kicad-developers team mailing list archive

["kristoffer Ödmark" <kristofferodmark90@xxxxxxxxx>]

I like the idea of using something as Protobuf and I agree fully with 
the benefits, especially since one can add/remove fields with minimal 
impact.

Basically the S-expression system used now is looking very much like a 
reinvented XML to me anyway, and storing protobuf-defined stuff as XML 
or similar seems actually nice.
There is one catch, and that is that we have to support opening a newer 
file, in an old software, and then store it again, without losing data 
that the software is not aware of. Or implement a way of not being able 
to store values in older software, when they open something newer.

There is also a middle way here, and that is to actually implement a 
Protobuf to S-Expression decoder/encoder, with the real benefit of 
actually defining fields in a modern well-known way, where the 
specification and implementetation does not have to manually be synced 
in code, comments, and a google doc. I have yet to see anything actually 
stay synchronized in such a manner over time, and many bugs manifest 
themself in these synchronization attempts. Anyway to avoid having to 
change the file-format another time, or add extra files to the side, I 
think that using an IDF is great next-step, mostly since the tooling, 
libraries and workflows for these are better defined.

But to be honest, I have a hard time understanding why we have to stick 
to the KiCad S-Expression, when there are quite readable text-formats 
that are widely supported already.

I know the requirement for the file format is readability, but I have 
yet to find and editor that actually understands the KiCad S-Expression 
(I have not searched extensively), but JSON,XML,YAML are usually read 
just fine, with syntax highlighting out of box. And an IDF would make 
these discussions quite reduntant, since changing file formats would be 
a minimal change in code, and not as now, where it is actually quite 
time-consuming.

- Kristoffer

On 2019-01-02 01:37, Andrew Lutsenko wrote:
> Hi Wayne,
>
> I would like to take this opportunity to do an elevator pitch for idea 
> of using one of IDL languages widely accepted in the industry like 
> Apache Thrift or Google Protobufs to define formats in KiCad.
> There are few large benefits in favor of using such languages:
>
> 1. They are self documenting. No more keeping a google doc in sync 
> with sources.
> 2. They are easily extensible. Just add a field, old parsers will 
> ignore it, new ones will pick it up. Need to deprecate a field? Just 
> add it's ID to reserved list to never reuse it again.
> 3. They have code generators for pretty much all commonly used 
> languages. That means anyone can pick KiCad file and just parse it in 
> Java/Go/Haskell or whatever language they fancy without porting over 
> s-expressions library or meticulously studying the file format doc. 
> This opens lot's of possibilities for third party tools to be added to 
> KiCad ecosystem. Writing a web viewer for schematic/pcb would be a 
> piece of cake for example.
>
> Other probably less impactful benefits:
> 4. Easy to serialize/encode in multiple formats. Need to send data 
> over network in compact form? No problem, just serialize using compact 
> binary protocol. Need to store in text file? just use text encoder.
> 5. Code generators will reduce amount of boilerplate in KiCad source. 
> Only actual application logic needs to be added on top of generated 
> data objects.
>
> I have personally worked extensively with both Thrift and Protobufs, I 
> think for KICad use case proto is better fit. Thrift has a lot more 
> library support for client/server RPC logic and defining RPCs is core 
> part of the language but we don't need any of that (at least for now). 
> Proto has all of that as extensions but it's core is just definition 
> of data types and it has better support for plain text format.
> Here are docs for both:
> https://developers.google.com/protocol-buffers/
> https://thrift.apache.org/tutorial/
>
> Let me know if any of that sounds interesting and if you have any 
> questions. I think this is worth investing time into and I'm willing 
> to help if needed.
>
> Regards,
> Andrew
>
> On Tue, Jan 1, 2019 at 11:59 AM Wayne Stambaugh <stambaughw@xxxxxxxxx 
> <mailto:stambaughw@xxxxxxxxx>> wrote:
>
>     I have updated and published the symbol file format[1] for comment.
>     Hopefully there isn't too much to change.  The only thing to really
>     finalize is the internal units.  The initial concept was unitless but
>     the more I think about it and discuss with other developers, it makes
>     more sense to use units for the following reasons:
>
>     1. It's easier to visualize in your head how the symbols on a
>     given page
>     size will layout.
>
>     2. Converting from other file formats (Eagle, Altium, etc) will be
>     easier since most if not all of them have a defined unit.
>
>     I'm thinking 10u (or possibly 100u) will make a good internal units
>     value.  Once we nail down the units, I will update the file format
>     document accordingly.
>
>     Please keep in mind that this is the symbol library file format
>     document
>     so things like constraints belong in the schematic file format.  I
>     will
>     be posting the schematic file format as soon as I finish updating it.
>
>     Cheers,
>
>     Wayne
>
>     [1]:
>     https://docs.google.com/document/d/1lyL_8FWZRouMkwqLiIt84rd2Htg4v1vz8_2MzRKHRkc/edit
>
>
>     _______________________________________________
>     Mailing list: https://launchpad.net/~kicad-developers
>     Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>     Unsubscribe : https://launchpad.net/~kicad-developers
>     More help   : https://help.launchpad.net/ListHelp
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp