kicad-developers team mailing list archive

[Dick Hollenbeck <dick@xxxxxxxxxxx>]

On 07/29/2010 11:58 AM, Brian Sidebotham wrote:
> On 29 July 2010 16:04, Dick Hollenbeck <dick@xxxxxxxxxxx> wrote:
>   
>> Jean-Pierre,
>>
>> First, I apologize for saying the BOM C++ code is garbage.  (I had my
>> special hat on,  :) you know the one that says any code I have not
>> written is garbage.  It's a
>> silly self-misleading hat, but I sleep in it.)
>>
>>
>> Does the output of netform.cpp's Write_GENERIC_NetList() have what is
>> needed in it to create a BOM?
>>
>>
>> If so, here is my current thinking, obviously it is evolving as I do the
>> research, before the development.
>>
>> What we need is almost entirely present in the existing *generic netlist
>> exporter*. (EESCHEMA -> Netlist -> Add Plugin).
>>
>> This can export the netlist to a text file and chain load a post
>> processor to generate a "netlist" of any kind.  Is there a reason this
>> cannot generate a BOM?  The only netlist plug-in in existence is in
>> eeschema/plugin/netlist_form_pads-pcb.cpp, and this represents the only
>> dependency on the existing non-XML "generic netlist format", which is an
>> arbitrary format that I would like to change to XML.
>>
>>
>> What about the following plan of action, for which I am volunteering:
>>
>>
>> 1) Change the generic netlist exporter to export to XML, keeping all the
>> existing content, even pins, and *adding* all custom component
>> fieldnames/properties.
>>
>> 2) Fix netlist_form_pads-pcb.cpp to read the new generic netlist format
>> XML file, using the xml support within wxWidgets, see wxXmlDocument, and
>> create the same output it uses now.  I'm hoping what was a non-gui
>> program can still stay that way using this class, don't know yet.
>>
>> 3) Work with Brian to write one or more python scripts, which will
>> basically fix bug https://bugs.launchpad.net/kicad/+bug/581153
>>     
>   
>> 4) Commit the C++ improvements I made yesterday to the existing BOM
>> generator and keep them until a higher authority deletes them.
>>
>> 5) I bail on fixing bug https://bugs.launchpad.net/kicad/+bug/581153 in
>> C++, which I deem being a lost cause, because I think the BOM generator
>> C++ is a lost cause.  Use any one of the python scripts to overcome it.
>>     
> This sounds like a sane workflow to me. Dick, please feel free to
> forward me example XML when you are ready.
>
> Best Regards,
>
> Brian.
>   

Thanks Brian.  I really appreciate your help.  I decided just to code up
the generic exporter using the wxXml library.  This is about 60% done,
and I should have you a file within a day.  That looked to be easier
than hand creating one.  All we have to do is populate and xml doc and
save it, and to restructure the document should be easier from the
source code.


We Jean-Pierre's help at the end, we can put everything in there but the
kitchen sink.  I will be committing my work within the next day to avoid
any merge conflicts.  As usual I went on code cleaning binge.

Your help Brian, is really really appreciated. I'm trusting you to
figure out the python XML handling. 

As an aside, and a reminder, I said before that in any situation where
there is no collating required, xsltproc is very simple to use.  Jean
Pierre most all the netlist formats could be done using it if need be.

By collating, I mean gathering up like part numbers and putting them on
a single line. I don't know XSLT well enough to say how hard that is,
but the non-collating outputs are extremely easy using XSLT.  And
require very few lines of XSLT code.  I taught myself the thing earlier
this week and have XSLT conversions in the field already.  There was a
nice XSLT tutorial out there, interactive on the web.

Brian, I have some BOM Java code that can do both collating and
reconciling of dissimilar field names among components.  Is that of
interest to you?  We need to support a couple different CSV outputs, a
simple one with a single part per row.  Then the granddaddy collated one
that has these characteristics:

0) column titles in first row.

1) collated output, one part number with potentially multiple
reference-ids per row.  The reference-ids, as a whole list should be
wrapped in double quotes, but you can put commas between the
reference-ids.  This way that whole list is loaded as a single cell into
the spreadsheet later.

2) fieldname columns should be added dynamically as you find them among
the components.  This means doing it in a two pass scenario, first pass
gathers all the unique fieldnames.

3) You may want to support a list of fieldnames that you "care" about
near the top of the script, so if a fieldname is not among them, it is
not output into the CSV.  This included list can be directly editable
within the *.py file for all I care.  Or you can handle a configuration
file if you want to impress somebody (I would not do it.)


The Java code I wrote first outputs a single reference-id per row table
and then the collated table all in the same CSV.  This way you have two
regions of the spreadsheet later after loading it into the spreadsheet. 
The Java code got its input from parsing the *.SCH file directly, and
although it handles hierarchical sheets, it did not handle the
multi-instancing of a particular sheet. The XML file you will be working
with as input will have that taken care of already.


Dick