set - Python - Iterating through two files to create a new file that has fields from second file appended to fields of first file -
new python attempted use logic answers in @mgilson
, @endolith
, , @zackbloom
zack's example
- i getting bunch of blank columns placed in front of first field of primary record.
- my out_file empty (more because of columns 2 files cannot match up.
how can fix this? end result should following:
('pudo_id','load_id','carrier_id','pudo_from_company','pudoitem_id';'pudo_id';'pudoitem_make') ('1','1','14','fmh material handling solutions','1','1','crown','tr3520 / twr3520','tuggers') ('2','2','7','wiese usa','2','2','cat','ndc100','3','2','cat','ndc100','4','2',' 2 batteries')
note: in output of 3rd row, appended 3 rows sub file array, while first 2 rows appended 1 row sub file. determined value in pri[0] , sub[1] comparing true.
here code based on @zack bloom
:
def build_set(filename): # set stores collection of unique items. both adding items , searching them # quick, it's perfect application. found = set() open(filename) f: line in f: # tuples, unlike lists, cannot changed, requirement # being stored in set. line = line.replace('"','') line = line.replace("'","") line = line.replace('\n','') found.add(tuple(sorted(line.split(';')))) return found set_primary_records = build_set('c:\\temp\\oz\\loads_pudo.csv') set_sub_records = build_set('c:\\temp\\oz\\pudo_items.csv') record = [] open('c:\\temp\\oz\\loads_pudo_out.csv', 'w') out_file: # using open files ensures closed, if code # raises exception. pri in set_primary_records : sub in set_sub_records : #out_file.write(" ".join(res) + "\n") if sub[1] == pri [0] : record = pri.extend(sub) out_file.write(record + '\n')
sample source data (primary records):
pudo_id;"load_id";"carrier_id";"pudo_from_company" 1;"1";"14";"fmh material handling solutions" 2;"2";"7";"wiese usa"
sample source data (sub records):
pudoitem_id;"pudo_id";"pudoitem_make" 1;"1";"crown";"tr3520 / twr3520";"tuggers" 2;"2";" cat";"ndc100" 3;"2";"cat";"ndc100" 4;"2";" 2 batteries" 5;"11";"midland"
the extend attribute not available tuples build_set creating. tuples immutable can concatenated or sliced normal python string functions.
for example:
with open('c:\\temp\\oz\\loads_pudo_out.csv', 'w') out_file: pri in set_primary_records : sub in set_sub_records : if sub[1] == pri[0] : record = pri + sub out_file.write(str(record)[1:-1] + '\n')
this same code above, modified allow tuple concatenation. in write line convert record string , strip start , end brackets, before appending '\n'. maybe there better / prettier ways this, i'm new python too.
edit: output expecting, few changes required:
# on line, remove sort() not wish change tuple item order.. found.add(tuple(line.split(';'))) ... open('c:\\temp\\loads_out.csv', 'w') out_file: pri in set_primary_records: record = pri # record tuple set in main loop sub in set_sub_records: if sub[1] == pri[0]: record += sub # each match, sub appended record out_file.write(str(record) + '\n') # removed stripping of brackets
Comments
Post a Comment