kicad-developers team mailing list archive

[Dick Hollenbeck <dick@xxxxxxxxxxx>]

On 10/06/2010 06:47 PM, Wayne Stambaugh wrote:
> On 10/6/2010 6:28 PM, Dick Hollenbeck wrote:
>   
>> On 10/06/2010 03:27 PM, Wayne Stambaugh wrote:
>>     
>>> On 10/6/2010 3:17 PM, Dick Hollenbeck wrote:
>>>   
>>>       
>>>> On 10/05/2010 07:04 PM, Wayne Stambaugh wrote:
>>>>     
>>>>         
>>>>> On 10/5/2010 10:33 AM, Dick Hollenbeck wrote:
>>>>>   
>>>>>           
>>>>>>             
> <<< snipped >>>
>
>   
>>>>     
>>>>         
>>> I wonder if it would be beneficial to create a LIBRARY_SOURCE for reading the
>>> existing library file format to maintain backwards compatibility.  The source
>>> code already exists to read these files.  Granted you wouldn't get any of the
>>> fancy new features like inheritance but you at least would not loose your
>>> current investment.  I don't know that an end around the LIBRARY_SOURCE API
>>> would be necessary.  I'm sure you could create a LIBRARY_SOURCE object that can
>>> provide writing so a derived class like OLD_KICAD_LIBRARY_SOURCE could even
>>> write library files in the old format ( not that you would want to ).
>>>   
>>>       
>>
>> I'm not a fan of adding write access functions to the LIBRARY_SOURCE
>> abstraction, not in any case.
>> I think you need to do an end run, there's only one special case of a
>> library source that is writable.
>>
>> This is where I claim KISS.
>>     
> Fair enough.  I was just thinking if there was one case where we would
> want to write there may be another one that we haven't considered at
> this point.  I trust your judgment on this one.
>
>   
>> Moving on to what the LIBRARY_SOURCE has to do, it has to give you the
>> sexpr text in the COMPONENT or symbol, shown below.
>>
>> class COMPONENT or SYMBOL (i.e. CODE_FRAGMENT)
>> {
>>     pins              // pin table
>>
>>     properties        // aka fields collection
>>
>>     graphics          // graphics primitives
>>
>>     std::string    sexpr;
>> }
>>     
> My original thinking was that SYMBOL would be one level lower in the
> food chain and not contain any properties.  The COMPONENT object could
> include one or possibly more SYMBOLs.
>
> class SYMBOL
> {
>     pins                // pin table
>     graphics            // graphics primitives
>
>     std::string         // sexpr
> }
>
> class COMPONENT
> {
>     symbol              // symbol table
>
>     properties          // field table.
>
>     std::string         // sexpr
> }
>
>   

I like that.  Here's a subtle twist on the idea:  A COMPONENT could
simply lose the symbol table and replace it with pins & graphics
containers (= table = list).  Then during the parsing, when encountering
the
symbol's (pin) and (line) tokens, the parser simply puts those into the
including COMPONENT's pin and graphics containers additively, as C++
binary objects of course.  This way you only have one pin containere
when you are all done, and one graphics container in the including
COMPONENT.  This makes hit testing, binary searching, and drawing easier.

But you can still support multiple symbols, providing they can be
visually coalesced in a meaningful way and don't need their own
graphical instantiation offsets.

But by allowing only one symbol you can forget about the origin
discrepancy.  The one and only "base" symbol sets that.  You should also
of course be able to extend another component, and that component should
be able to be extended also, etc.  

I would lean towards a single inheritance model myself, and like the
single pin and graphics container idea better.  It also has the
advantage of a single search space for things like (pin_del 12).  If
done this way, then take another look at the C++ structures with this
modification, and then I don't know that you need a separate class for
SYMBOL?  What do you think?  We could simply stop using the term also,
and in our discussions call it a 'base component' or 'graphics
component' or ________?  Open to ideas on that one.


>>
>> The sexpr is the source to the component.  So unless you can load your old format components and create the new source file for it, I'd say this is wasted time.    Its a mess
>>
>> IMHO, it is better to run a batch file and convert everything up front one time.  
>> Somebody can still use the old version with the old software.
>>
>>
>> class LIBRARY
>> {
>>     COMPONENTS  collection;  some have been loaded, some are simply
>> known to be available on a deferred basis.
>>
>>     LIBRARY_SOURCE* loader;
>> }
>>
>>
>> LIBRARY_SOURCE* is simply a text file fetcher, and by that I mean a
>> std::string reader just like you read a HTML page before you display it
>> in a browser.
>>
>> class LIBRARY_SOURCE
>> {
>>
>>     virtual std::string Read( GUID )
>>     :
>>     :
>>     list directory
>>
>>     list directory on a component.  the final path fragment is the
>> version number.
>>
>>     search for name value pairs in the properties
>> }
>>
>>
>> LIBRARY_SOURCE::Read() just reads a blob of text that is sexpr, and you
>> have to save that in COMPONENT::sexpr, at some point it will be parsed. 
>> It might not be a full component, so parsing it outside of a component
>> might not make sense until it is inherited by something that inherits or
>> includes it.  You might parse it just to find any nested inherits or
>> includes, such that those portions can also be cached in their
>> respective LIBRARYs.
>>
>>
>> Finally, parse the top most goody with STRING_LINE_READER and DSNLEXER
>> into the top most including COMPONENT.
>>
>>
>> I might base the LIBRARY_SOURCE API on a subset of SFTP like functions,
>> limited to read only access.  It is sort of a virtual file system, but
>> dumber, since you only need read only access, directory listing, and
>> something that lets you search.  You can pull it off using http protocol
>> behind the scenes,
>>
>> When talking to the server, on the other side of the world, you do not
>> need to know you are using a SCCS because the revision could be like a
>> final path component in the GUID.  Or something like a file extension:  
>> .456
>>
>>
>> As far as the server goes:
>>
>> The server accesses the SCCS and caches the part by revision ID, say in
>> a hashtable.  It responds to http protocol requests.  Because the
>> server's commitment is to honor the same sexpr text for any GUID, it in
>> theory never has to flush its cache.  New commits to the SCCS by the
>> librarians go in as a newer version of a part and not actual
>> replacement.  Like a webpage, and because of inclusion, you may have to
>> hit various LIBRARY_SOURCEs multiple times get everything into your
>> local LIBRARYs in RAM.
>>
>> So for the remote case:
>>
>> EESCHEMA -> LIBRARY -> LIBRARY_SOURCE -> http -> SERVER, say in python
>> or java or apache using websvn derivative -> SCCS
>>
>> An calming consideration in all this is that much of what we are dealing
>> with here is read only, cached data, and it eventually ALL finds it way
>> into one of the open LIBRARYs (caches).  So it is cached in our local
>> LIBRARY and if remote, also in the server.  With a goofy query string
>> the server could give you SVG instead of sexpr so you can actually hit
>> it with a web browser also and see what you are looking at.
>>
>> To do the SVG form, the server would have to in line expand all the
>> inheritance, and this would require that the repo not reference anything
>> outside itself.
>>     
> This concept has interesting implications for the 3D models in the PCB
> editor as well.
>   

For 3D there is a new web version of opengl in HTML5.  Needs to be
looked at for this.  In fact HTML5 has this <canvas> thing that might be
better for 2D than SVG also.


>   
>>     
>>>   
>>>       
>>>> To test whether you get what I am saying, and fully understand the
>>>> concept of a parts list, it should be noted that I did not list "parts
>>>> list" as a "library source" for the schematic in which it resides.  A
>>>> parts list role is very unique with respect to the schematic in which it
>>>> resides.  It is the instantiation factory for that schematic and no
>>>> others.  However it can act as a library source for other schematics. 
>>>> This design will keep the parts list complete within every schematic,
>>>> and every schematic will stand alone, now and forever.  :) But this
>>>> assumes the inherited components' library sources remain accessible
>>>> forever, and with a GUID that includes a version number, and with a
>>>> stable remote library source, I think it could actually happen that way.
>>>>     
>>>>         
>>> If we could pull this off, that would be excellent.  One of the never ending
>>> headaches with every CAD system I have ever used is libraries.  It would be
>>> great if I could copy a project folder from one computer to another without
>>> having to remember to copy all of the library dependencies to each computer.
>>>
>>>   
>>>       
>>>> The notion of loading needs to be better defined, since *deferring* may
>>>> be desired in the remote case.  There should be a "get contents"
>>>> (directory listing) function with a component specific load after that. 
>>>> Otherwise the remote library sources could be too slow.  Some more
>>>> thought needs to be put into this.
>>>>     
>>>>         
>>> The remote case is going to be the most difficult one to solve. 
>>>       
>> Not by much.  Not if the APIs are designed properly up front, the
>> abstraction makes the client code identical.
>>
>> The server does caching too, and I think a good sized server would have
>> all the components cached in RAM anyway.  It would be like hitting a
>> website that has all the pages already cached in RAM.  And if we use
>> http protocol for this, it is very do-able.
>>
>>
>>     
>>>  I suppose you
>>> could check to see if the local copy of the library repo is up to date with the
>>> remote repo if remote access is available and update the local repo when
>>> necessary.  You would always use the local copy of the repo when editing a
>>> schematic to prevent the latency of accessing the remote repo.
>>>   
>>>       
>> For my personal use, I would not even feel compelled to copy the repo. 
>> It is "out there" and ready, on the net.  A typical component would have
>> what, less than 300 character's in the http content part of the reply?
>>     
> My concern is when you do not have access to "out there".  There are
> many reasons for this so it would be nice to have the option of keeping
> a local copy to fall back on for those times when you don't have net access.
>   

Well there were 5 kinds of LIBRARY_SOURCEs mentioned, and this need was
covered by one.


> All in all it seems like a very sound concept.  It will be interesting
> to see how it works in practice.
>   

I had most of these ideas about 10 months ago, and I see it coming
streaming back into my head now.  At that time it was in reference to
footprints.  But with respect to the libraries, its all pretty much the
same.  Only the instantiation aspect of components vs. footprints in
schematics or boards respectively, is different.  In PCBNEW the
instantiation is initially a full on copy of the footprint, whereas in
EESCHEMA the instantiation is only a pointer to something in the parts
list.  And that does not affect the library concepts being discussed here.

Common to both areas are:

1) LIBRARY_SOURCE should or could be basically identical.

2) LIBRARY, with the twist that one holds footprints, one holds components

3) a container to hold the LIBRARYs and this is where I see the LIBRARY
routing caused by the GUID taking place, in part.

4) A couple of lookup tables for library URIs.


Maybe we do a series of *.h files with Javadoc comments describing the
classes and functions using comments and function declarations.  We can
generate html from that, and if it looks sound, folks could starting
coding implementation functions and classes.

So back in December of last year I had mentioned I would do the *.h
files, maybe real soon now is the best time.  Right after a good night's
sleep and lots of coffee.  I could check in daily and others could add
their own edits and attach patches to email postings.  This way more
than one of us could churn.  We can put everything in one *.h file for
now, and generate javadoc html from that.  Later we can split up the
file.  BTW I would want to throw exceptions from the LIBRARY_SOURCE
functions and keep the UI out of there.

Jean-Pierre, do you think this is worth my time? 

I must confess that I am drawing a blank on 3d models in this
discussion.  I will not be able to contribute any ideas on those, for
now.  Somebody can come in later and dovetail them in.  For now it is
more than I care to discuss.

Dick