kicad-developers team mailing list archive
-
kicad-developers team
-
Mailing list archive
-
Message #14397
Feature request: Extended BOM (Part, Footprint, etc.) management
Hello!
This is my first post to this mailing list and I wasn't following recent
development in other posts so I don't know if something similar is
already in planning.
Last autumn my problem was, that I had a quite big project with multiple
sub-schematas and stuff and wanted it to be manufactured externally.
Fabs always want a nice fabs for the parts to use and sometimes it's
better/cheaper to use similar parts.
For example you need 4 SM0603/2uF capacitors and one SM0402/4uF
capacitor you may want to use 5 of either because it doesn't really
matter. Now it's really hard to find the parts you want to change, you
have to generate a BOM find the part-number, search it in the schemata
change the uF value, then export the new netlist, search the part again
in the CvPcb and edit the footprint for each part.
Also if you want to change some other field it would be much easier to
just edit it in a spreadsheet application like excel or so.
Because of that I hacked together some (very ugly!) python scripts with
my own (even uglier!!) eeschema-parser (Kicad_schema_parser.py).
Now the process for editing fields looks like this :
Generate_BOM_from_N-Schematas.py
- ./Generate_BOM_from_N-Schematas.py MyTopSchema.sch
- this generates a MyTopSchema_BOM.csv file from MyTopSchema.sch and all
it's sub-schematas
- it only exports fields specified in the field_names-variable of the
parts class-instance in Generate_BOM_from_N-Schematas.py
parts=Parts(
field_names=[
"Value",
"Footprint",
"Type",
"Voltage Rating",
"Manufacturer",
"Part Number",
"Comments"
]
)
- if a field isn't specified for a part it will be empty in the csv
- the rows are sorted by 'Chip Name', then by field_names
("Value",Footprint",etc..)
- the fields 'Chip Name', Count and References will always be
generated (and exported)
- Count is only informative, the count of space separated references
in the References row
- References are grouped if they have exactly the same field-values
(which is not so good if not even 'Value' is specified)
I think you can also add new columns but I'm not quite sure
- Now I edit the csv, fill in footprints/manufacturers/etc. where
applicable
After editing the BOM i save it again in the same csv-file (also same
format!)
Add_user_fields_to_components.py
- /Add_user_fields_to_components.py -BOM=MyTopSchema_BOM.csv
MyTopSchema.sch
- this changes the fields specified in the third line of
MyTopSchema_BOM.csv
- if a field is empty it is deleted
- if a field is not specified on row 3 it's left unchanged, maybe :)
Reopen MyTopSchema.sch with eeschema.
Export the netlist
Run CvPcb, save and close CvPcb
Update_Footprints_in_CMP.py MyTopSchema.sch
- ./Update_Footprints_in_CMP.py MyTopSchema.sch change
- this changes the footprints in MyTopSchema.cmp according to the
footprint field
Run Pcbnew
Import netlist with change/replace footprints and so..
Fertig!
Would it be hard to implement something similar directly in Kicad?
I think the first thing to change is the BOM-export form. Or maybe it
would be better if there was a project based user-field-manager or so..
Cheers,
Oli
PS:
Most of the scripts have 'help-notices' if you run them without
arguments. If your interested in the scripts, I could clean them up
for you. If you want to try them out make backups first!
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from sys import argv
from os.path import basename, splitext, isfile, isdir, join
from operator import itemgetter
from pprint import pprint
from Kicad_schema_parser import *
chip_name = ""
alterations = []
files = []
try :
chip_name = "*"
argv = split_save_str((" ".join(argv)).strip(), ' ')
if argv[1][:len("-BOM=")]=="-BOM=" :
alterations = BOM_to_alterations(argv[1][len("-BOM="):])
if alterations is None or len(alterations)==0 :
raise Exception("no alterations found in BOM")
file_idx=1
else :
chip_name = clean_str(argv[1])
file_idx = argv.index("-")
for f in argv[2:file_idx] :
f=clean_str(f)
f = f.split("?")
if len(f) <> 2: raise Exception("malformed expression "+str(f))
conditions = {}
for f0 in f[0].split("&&") :
f0=f0.strip()
if not f0 : continue
c = f0.split("==")
if not len(c)==2: raise Exception("malformed condition "+str(f))
conditions[clean_str(c[0])] = clean_str(c[1])
changes = {}
for f1 in f[1].split(";") :
c = f1.split("=")
if len(c) <> 2: raise Exception("malformed field-assignment "+str(f))
changes[clean_str(c[0])] = clean_str(c[1])
if len(changes)==0: raise Exception("shit! no changes!")
alterations.append((conditions,changes))
files = argv[file_idx+1:]
for f in files[::] :
if not isfile(f) : raise Exception("shit! not a file")
if len(files)==0 : raise Exception("specify at least one file")
except Exception, ex:
print ex.message
print
print "usage A : Add_user_fields_to_components.py [Chipname] [F==V [&& F==V].. ].. ? F=V[;F=V].. - [filepath].."
print "eg. ./Add_user_fields_to_components.py C \"Value==100n?Voltage=10V;Type=X5R;\" - file1.sch file2.sch"
print "=> where Chipname is 'C' and Value=='100n'"
print " set/create field Voltage='10V' and field Type='X5R'"
print
print "usage B : Add_user_fields_to_components.py -BOM=BOM-File.csv [filepath].."
exit(0)
processed_files=[]
while len(files) :
f = files.pop()
skip=False
for p in processed_files :
if f.find(p)>-1 :
print "duplicated file %s"%p
skip=True
break
if skip : continue
processed_files.append(basename(f))
print "changing",f
data = open(f,'r').read().split("\n")
nf=""
part=None
is_in_comp = False
is_in_sheet = False
for l in data :
if l=="$Comp" :
is_in_comp=True
part = Part()
elif l=="$EndComp" :
part.feed_comp_line(l)
if (part.chip_name==chip_name or chip_name=="*") and len(part.fields) :
part.alter_fields(alterations)
nf += part.get_component_string()
part = None
is_in_comp=False
continue
elif l=="$Sheet" :
is_in_sheet=True
elif l=="$EndSheet" :
is_in_sheet=False
if is_in_comp :
part.feed_comp_line(l)
continue
elif is_in_sheet and len(l)>3 and l[:3]=="F1 " :
files.append(clean_str(split_save_str(l.strip()," ")[1]))
nf += l+"\n"
open(f,'w').write(nf.strip())
print "done."
#!/usr/bin/env python
from sys import argv
from os.path import basename
import re
print "usage : ./Change_Sheet-IDs.py add[:min] file1-without.type file2-without.type ..."
v=argv[1].split(":")
shift=int(v[0])
min_val=0
if len(v)>1 : min_val=int(v[1])
pot=0
s=abs(shift)
while s :
pot+=1
s/=10
print shift, min_val, pot
def update_file(f, searchstr,endel) :
global pot, shift, min_val
data = open(f,'r').read()
nf = open(basename(f)+".edit",'w')
x=0
pot_bkp = pot
while 1 :
off=x
y=data[x:].find(searchstr)
if y<0 : break
x+=y
x+=len(searchstr)
y=data[x:].find(endel)
if y<0 : break
x+=y
while pot>2 :
try :
y=x-pot
val=int(data[y:x])
break;
except :
pot-=1
if pot==2 : exit(1)
if val>=min_val : val+=shift
nf.write(data[off:y]+str(val))
#break
nf.write(data[off:])
nf.close()
for f in argv[2:] :
update_file(f+".kicad_pcb"," reference "," ")
update_file(f+".cmp","Reference = ",";")
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from sys import argv
from os.path import basename, splitext, isfile, isdir, join
from operator import itemgetter
from pprint import pprint
from Kicad_schema_parser import *
ref = ""
alterations = []
files = []
try :
if len(argv)<2 or len(argv)>2 : raise Exception("check usage..")
start_file = argv[1]
if not isfile(argv[1]) : raise Exception("not a file")
except Exception, ex:
print ex.message
print
print "usage : ./Generate_BOM_from_N-Schematas.py [filepath]"
print "eg. ./Generate_BOM_from_N-Schematas.py file1.sch"
exit(0)
files = [ start_file ]
parts=Parts(
field_names=[
"Value",
"Footprint",
"Type",
"Voltage Rating",
"Manufacturer",
"Part Number",
"Comments"
]
)
processed_files=[]
while len(files) :
f = files.pop()
skip=False
for p in processed_files :
if f.find(p)>-1 :
print "duplicated file %s"%p
skip=True
break
if skip : continue
processed_files.append(basename(f))
print "collecting parts from",f
data = open(f,'r').read().split("\n")
part=None
is_in_comp = False
is_in_sheet = False
for l in data :
if l=="$Comp" :
is_in_comp=True
part = Part()
elif l=="$EndComp" :
part.feed_comp_line(l)
parts.add_part(part)
# only dif from "Add_user_field_loop.."
#if (part.chip_name==chip_name or chip_name=="*") and len(part.fields) :
# part.alter_fields(alterations)
#nf += part.get_component_string()
part = None
is_in_comp=False
continue
elif l=="$Sheet" :
is_in_sheet=True
elif l=="$EndSheet" :
is_in_sheet=False
if is_in_comp :
part.feed_comp_line(l)
continue
elif is_in_sheet and len(l)>3 and l[:3]=="F1 " :
files.append(clean_str(split_save_str(l.strip()," ")[1]))
open(splitext(start_file)[0]+"_BOM.csv",'w').write(str(parts))
print parts.num_parts, "parts found."
print "done."
# -*- coding: utf-8 -*-
import re
MANDATORY_FIELD_NAMES = ["Reference","Value","Footprint","Datasheet"]
def clean_str(f) :
f=f.strip()
if f=='"' or f=='' : return ''
s=0;e=0
if f[0] =='"' : s=1
if f[-1]=='"' : e=1
if (s+e) :
f=f[s:-e]
return f
def save_str(f) :
return '"'+str(f).replace('"','')+'"'
#def split_save_str(s,d) :
#first = True
#save_part = False
#r=[]
#for x in s.split('"') :
##x = x.strip()
#if save_part :
#r.append(save_str(x))
#elif len(x)>int(not first) : r += x.split(d)[:-1]
#save_part = not save_part
#first=False
#return r
def split_save_str(s,d) :
save_part_start = s.find('"')
if save_part_start==-1 : return s.split(d)
else : save_part_end = s.find('"',save_part_start+1)
current_offset = 0
r = []
next_split = s.find(d,current_offset)
while current_offset<len(s) :
if next_split == -1 :
r.append(s[current_offset:])
break
elif next_split < save_part_start or save_part_start==-1 :
r.append(s[current_offset:next_split])
if next_split==len(s)-1 :
r.append("")
break
current_offset = next_split+1
next_split = s.find(d,current_offset)
elif next_split < save_part_end :
next_split = s.find(d,save_part_end+1)
elif save_part_start<>-1 :
save_part_start = s.find('"',save_part_end+1)
if save_part_start<>-1 :
save_part_end = s.find('"',save_part_start+1)
if save_part_end==-1 : #bzw. next_split=-1
r.append(s[current_offset:])
break
else : save_part_end = -1
return r
def ref_cmp(a,b) :
a=int(re.search('[0-9]*$',a).group(0))
b=int(re.search('[0-9]*$',b).group(0))
if a==b : return 0
if a<b : return -1
return 1
def extract_value_field(part) :
value=part.fields[1][1] # see MANDATORY_FIELD_NAMES value index..
m=re.search('^[0-9.,]*',value)
i = m.end()
if i>0 :
v=float(m.group(0))
if i==len(value) : return v,""
p=value[i]
if p=='f': v/=1000000000000000
elif p=='p': v/=1000000000000
elif p=='n': v/=1000000000
elif p=='u' or p=='µ': v/=1000000
elif p=='m' : v/=1000
elif p=='k' : v*=1000
elif p=='M' : v*=1000000
elif p=='G' : v*=1000000000
elif p=='T' : v*=1000000000000
else : i-=1
value=value[i+1:]
else : v=0
return v,value
def part_cmp(a,b) :
if not (a.complete and b.complete) : raise Exception("Part not complete")
if a.chip_name<b.chip_name : return -1
if a.chip_name>b.chip_name : return 1
a_v, a_value = extract_value_field(a)
b_v, b_value = extract_value_field(b)
if a_v<b_v : return -1
if a_v>b_v : return 1
if a_value<b_value : return -1
if a_value>b_value : return 1
return 0
#pprint( alterations )
class Part :
def __init__(self) :
self.chip_name=""
self.references=[]
self.fields={}
self.comp =""
self.complete = False
self.shit=False
def feed_comp_line(self,l) :
if self.complete : raise Exception("Part already complete")
if not len(l)>2 : return
fitems = split_save_str(l.strip()," ")
if fitems[0]=='L' :
self.chip_name=fitems[1]
elif fitems[0]=='AR' :
self.add_reference(clean_str(fitems[2].split("=")[1]))
#print "dropping AR-FIELD" ; return
if fitems[0]=='F' :
if len(self.fields)==0 : self.comp+=">>>fields<<<\n"
self.add_field(fitems)
else :
self.comp+=l+"\n"
if l=="$EndComp" :
self.complete=True
def get_component_string(self) :
if not self.complete : raise Exception("Part not complete")
fields = ""
for fitems in self.fields.itervalues() :
fields+=" ".join(fitems[2])+"\n"
fields=fields.strip()
return self.comp.replace(">>>fields<<<", fields)
def add_reference(self,ref) :
try : self.references.index(ref)
except : self.references.append(ref)
def add_field(self,fitems) :
fi = int(fitems[1])
if fi<len(MANDATORY_FIELD_NAMES) : name = MANDATORY_FIELD_NAMES[fi]
else :
name = clean_str(fitems[-1])
if name=="Comment" :
name="Comments"
fitems[-1] = save_str(name)
print "Changing field name 'Comment' to 'Comments'"
value = clean_str(fitems[2])
if value=="~" :
value=""
fitems[2]='""'
if fi>3 and value=="" : return # remove empty fields (except mandatory fields)..
if fi>1 : # hide all fields, except "Reference","Value" (leave those untouched..)
fitems[8]="0001";
if fi==0 :
self.add_reference(value)
self.fields[fi] = (name,value,fitems)
def get_field(self, field_name) :
fi = -1
try : return self.fields[MANDATORY_FIELD_NAMES.index(field_name)]
except : pass
try :
for k,f in self.fields.iteritems() :
if k<len(MANDATORY_FIELD_NAMES) : continue
if f[0]==field_name : return self.fields[k]
except : pass
return None
def alter_fields(self,alterations) :
for alter in alterations :
cond = True
for key,value in alter[0].iteritems() :
for fi,finfo in self.fields.iteritems() : # match the field name
if finfo[0]==key : #
cond &= finfo[1]==value # update the condition
break
if cond :
for key,value in alter[1].iteritems() :
field_existing = False
changed = False
remove_fields=[]
for fi,finfo in self.fields.iteritems() : # match the field name
if finfo[0]==key : #
field_existing = True
if value<>finfo[1] :
if fi>=len(MANDATORY_FIELD_NAMES) and value=="" :
# removing the field
print "removing field[%d] %s (%s)" %(fi,finfo[0]," ".join(self.references))
remove_fields.append(fi)
#finfo[2][2]=save_str(value) # changed for field
else :
# set the new value
print "changing field[%d] %s: %s -> %s (%s)" %(fi,finfo[0],finfo[1],value," ".join(self.references))
#finfo[1]=c[1] #unchanged for cond
finfo[2][2]=save_str(value) # changed for field
changed=True
break
for fi in remove_fields : self.fields.pop(fi)
if not field_existing and value<>"" : # add a new field
#pprint( cfields )
keys = self.fields.keys()
keys.sort()
fi_l = keys[-1] # find the last field
finfo_l = self.fields[fi_l]
fi = fi_l+1
fitems = finfo[2][::]
fitems[1]=str(fi)
fitems[2]=save_str(value)
if fitems[-1]=="CNN" : fitems.append(save_str(key))
elif fitems[-2]=="CNN" : fitems[-1]=save_str(key)
else : raise Exception("Wrong field? " + " ".join(fitems))
fitems[-4] = "0001" # make field invisible..
print " adding field[%d] %s: %s (%s)" %(fi,key,value," ".join(self.references))
self.fields[fi] = (key,value,fitems)
#for finfo in self.fields.itervalues() :
# nf += " ".join(finfo[2])+"\n"
def compare(self,part) :
if not self.complete : raise Exception("Part not complete")
if not part.complete : raise Exception("Part not complete")
if len(self.fields)<>len(part.fields) : return False
for f in self.fields.itervalues() :
if f[0] == "Reference" : continue
found = False
for f2 in part.fields.itervalues() :
if f[0] == f2[0] and f[1] == f2[1] : found=True
if not found : return False
return True
def to_string(self, field_names=None):
self.references.sort(ref_cmp)
ret = '%s\t%d\t%s' %( save_str(self.chip_name),len(self.references),save_str(" ".join(self.references)) )
if field_names==None :
for f in self.fields.itervalues() :
if f[0] == "Reference" : continue
ret += '\t%s=%s'%(f[0],f[1])
else :
for fn in field_names :
ret += "\t"
for f in self.fields.itervalues() :
if f[0]==fn : ret += save_str(f[1])
return ret
def __str__(self):
if self.complete :
return self.get_component_string()
else :
return "incomplete part"
class Parts :
def __init__(self, field_names=[]) :
self.num_parts = 0
self.parts=[]
self.field_names=field_names
def add_part(self, part) :
if not part.complete : raise Exception("Part not complete")
for p in self.parts :
if p.compare(part) :
off=len(p.references)
for ref in part.references :
p.add_reference(ref)
self.num_parts+=len(p.references)-off
return
self.num_parts+=len(part.references)
self.parts.append(part)
for f in part.fields.itervalues() :
if f[0] == "Reference" : continue
if f[1]=="" : continue # skip fields with empty values (mostly the mandatory fields like Datasheet..)
try : self.field_names.index(f[0])
except : self.field_names.append(f[0])
def find_reference(self, reference) :
if reference=="" : return None
for p in self.parts :
for r in p.references :
if r==reference : return p
return None
def __str__(self) :
ret = "%d parts\n"%self.num_parts
ret+= '\n"Chip Name"\t"Count"\t"References"'
for f in self.field_names :
ret += '\t%s' %save_str(f)
ret+="\n"
self.sort()
for p in self.parts :
ret += p.to_string(self.field_names)+"\n"
return ret
def sort(self) :
self.parts.sort(part_cmp)
# conditions-format = {field_name: value}
# changes-format = {field_name: value}
# alteration-format = (conditions,changes)
from os.path import isfile
def BOM_to_alterations(f) :
if not isfile(f) : raise Exception("%s is not a file"% f)
print "BOM_to_alterations(%s)" %f
data = open(f,'r').read().split("\n")
alterations=[]
BOM_started = False
field_names=None
for l in data :
if l=="": continue
items=split_save_str(l,'\t')
if BOM_started :
if len(items)<>len(field_names) :
raise Exception("wrong item count in BOM!\n%s\n%s"%(str(items), str(field_names)))
changes={}
for i in range(3,len(items)) :
#if items[i]=="" : print "deleting field.."
changes[field_names[i]] = clean_str(items[i])
for reference in clean_str(items[2]).split(' ') :
alterations.append(({"Reference":reference},changes))
elif len(items) and clean_str(items[0])=="Chip Name" :
if len(items)<4 : raise Exception("too little information!")
BOM_started=True
field_names=[]
for f in items : field_names.append(clean_str(f))
return alterations
def walk_schema(f) :
print "tobedone.."
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from sys import argv
from os.path import basename, splitext, isfile, isdir, join
from operator import itemgetter
from pprint import pprint
def clean_str(f) :
return f.strip().replace("'",'').replace('"','')
def save_str(f) :
return '"'+f+'"'
files = []
try :
files = argv[1:]
for f in files[::] :
if not isfile(f) : raise Exception("shit! not a file")
if len(files)==0 : raise Exception("specify at least a file")
except Exception, ex:
print ex.message
print
print "usage : Remove_user_fields_from_components.py [filepath].."
exit(0)
FIELD_NAMES = ["Reference","Value","Footprint","Datasheet"]
cfields = None
for f in files :
c=-1
print "changeing",f
data = open(f,'r').read().split('\n')
nf=""
editing_fields = False
for l in data :
c+=1
if l[:5] == "F 0 \"" :
editing_fields=True
cfields = {}
if editing_fields and len(l) and l[0]=='F' :
fitems = l.split(" ")
fi = int(fitems[1])
if fi<len(FIELD_NAMES) :
name = FIELD_NAMES[fi]
value = clean_str(fitems[2])
cfields[fi] = (name,value,fitems)
else : print "dropping user field %s " %clean_str(fitems[-1])
continue
if not cfields is None :
for finfo in cfields.itervalues() :
nf += " ".join(finfo[2])+"\n"
cfields = None
editing_fields=False
nf += l+"\n"
open(f,'w').write(nf.strip())
print "done."
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from sys import argv
from os.path import basename, splitext, isfile, isdir, join
from operator import itemgetter
from pprint import pprint
from Kicad_schema_parser import *
ref = ""
alterations = []
start_file=""
cmp_file=""
save_changes=False
try :
if len(argv)<>3 : raise Exception("check usage..")
start_file = argv[1]
if not isfile(start_file) : raise Exception("not a file")
cmp_file = splitext(start_file)[0]+".cmp"
if not isfile(cmp_file) : raise Exception("not a file")
print cmp_file
if argv[2]=="dry_run" : save_changes=False
elif argv[2]=="change" : save_changes=True
else : raise Exception("check usage..")
except Exception, ex:
print ex.message
print
print "usage : ./Update_Footprints_in_CMP.py filepath dry_run|change"
print "eg. ./Update_Footprints_in_CMP.py file1.sch change"
exit(1)
files = [ start_file ]
parts=Parts(
field_names=[
"Value",
"Footprint",
"Type",
"Voltage Rating",
"Manufacturer",
"Part Number",
"Comments"
]
)
processed_files=[]
while len(files) :
f = files.pop()
skip=False
for p in processed_files :
if f.find(p)>-1 :
print "duplicated file %s"%p
skip=True
break
if skip : continue
processed_files.append(basename(f))
print "collecting parts from",f
data = open(f,'r').read().split("\n")
part=None
is_in_comp = False
is_in_sheet = False
for l in data :
if l=="$Comp" :
is_in_comp=True
part = Part()
elif l=="$EndComp" :
part.feed_comp_line(l)
parts.add_part(part)
# only dif from "Add_user_field_loop.."
#if (part.chip_name==chip_name or chip_name=="*") and len(part.fields) :
# part.alter_fields(alterations)
#nf += part.get_component_string()
part = None
is_in_comp=False
continue
elif l=="$Sheet" :
is_in_sheet=True
elif l=="$EndSheet" :
is_in_sheet=False
if is_in_comp :
part.feed_comp_line(l)
continue
elif is_in_sheet and len(l)>3 and l[:3]=="F1 " :
files.append(clean_str(split_save_str(l.strip()," ")[1]))
print parts.num_parts, "parts found."
data = open(cmp_file,'r').read().split("\n")
is_in_comp = False
reference = None
nf=""
for l in data :
if l=="BeginCmp" :
is_in_comp=True
elif l=="EndCmp" :
reference=None
is_in_comp=False
elif is_in_comp :
if l[:len("Reference = ")]=="Reference = " :
reference = l[len("Reference = "):-1]
elif l[:len("IdModule = ")]=="IdModule = " :
footprint = l[len("IdModule = "):-1]
part = parts.find_reference(reference)
if part==None :
exit(1)
f = part.get_field("Footprint")
if f==None or f[1]=="":
print "% 20s : has no footprint assigned!" %reference
elif f[1]<>footprint :
print "%20s : %20s > %20s" %(reference,footprint,f[1])
if save_changes :
l="IdModule = %s;" %f[1]
nf += l+"\n"
open(cmp_file,'w').write(nf)
print "done."
Follow ups
