← Back to team overview

kicad-developers team mailing list archive

Re: exploiting human readability

 

On 10/6/2010 6:28 PM, Dick Hollenbeck wrote:
> On 10/06/2010 03:27 PM, Wayne Stambaugh wrote:
>> On 10/6/2010 3:17 PM, Dick Hollenbeck wrote:
>>   
>>> On 10/05/2010 07:04 PM, Wayne Stambaugh wrote:
>>>     
>>>> On 10/5/2010 10:33 AM, Dick Hollenbeck wrote:
>>>>   
>>>>>

<<< snipped >>>

>>>     
>> I wonder if it would be beneficial to create a LIBRARY_SOURCE for reading the
>> existing library file format to maintain backwards compatibility.  The source
>> code already exists to read these files.  Granted you wouldn't get any of the
>> fancy new features like inheritance but you at least would not loose your
>> current investment.  I don't know that an end around the LIBRARY_SOURCE API
>> would be necessary.  I'm sure you could create a LIBRARY_SOURCE object that can
>> provide writing so a derived class like OLD_KICAD_LIBRARY_SOURCE could even
>> write library files in the old format ( not that you would want to ).
>>   
> 
> 
> I'm not a fan of adding write access functions to the LIBRARY_SOURCE
> abstraction, not in any case.
> I think you need to do an end run, there's only one special case of a
> library source that is writable.
> 
> This is where I claim KISS.

Fair enough.  I was just thinking if there was one case where we would
want to write there may be another one that we haven't considered at
this point.  I trust your judgment on this one.

> 
> Moving on to what the LIBRARY_SOURCE has to do, it has to give you the
> sexpr text in the COMPONENT or symbol, shown below.
> 
> class COMPONENT or SYMBOL (i.e. CODE_FRAGMENT)
> {
>     pins              // pin table
> 
>     properties        // aka fields collection
> 
>     graphics          // graphics primitives
> 
>     std::string    sexpr;
> }

My original thinking was that SYMBOL would be one level lower in the
food chain and not contain any properties.  The COMPONENT object could
include one or possibly more SYMBOLs.

class SYMBOL
{
    pins                // pin table
    graphics            // graphics primitives

    std::string         // sexpr
}

class COMPONENT
{
    symbol              // symbol table

    properties          // field table.

    std::string         // sexpr
}

> 
> 
> The sexpr is the source to the component.  So unless you can load your old format components and create the new source file for it, I'd say this is wasted time.    Its a mess
> 
> IMHO, it is better to run a batch file and convert everything up front one time.  
> Somebody can still use the old version with the old software.
> 
> 
> class LIBRARY
> {
>     COMPONENTS  collection;  some have been loaded, some are simply
> known to be available on a deferred basis.
> 
>     LIBRARY_SOURCE* loader;
> }
> 
> 
> LIBRARY_SOURCE* is simply a text file fetcher, and by that I mean a
> std::string reader just like you read a HTML page before you display it
> in a browser.
> 
> class LIBRARY_SOURCE
> {
> 
>     virtual std::string Read( GUID )
>     :
>     :
>     list directory
> 
>     list directory on a component.  the final path fragment is the
> version number.
> 
>     search for name value pairs in the properties
> }
> 
> 
> LIBRARY_SOURCE::Read() just reads a blob of text that is sexpr, and you
> have to save that in COMPONENT::sexpr, at some point it will be parsed. 
> It might not be a full component, so parsing it outside of a component
> might not make sense until it is inherited by something that inherits or
> includes it.  You might parse it just to find any nested inherits or
> includes, such that those portions can also be cached in their
> respective LIBRARYs.
> 
> 
> Finally, parse the top most goody with STRING_LINE_READER and DSNLEXER
> into the top most including COMPONENT.
> 
> 
> I might base the LIBRARY_SOURCE API on a subset of SFTP like functions,
> limited to read only access.  It is sort of a virtual file system, but
> dumber, since you only need read only access, directory listing, and
> something that lets you search.  You can pull it off using http protocol
> behind the scenes,
> 
> When talking to the server, on the other side of the world, you do not
> need to know you are using a SCCS because the revision could be like a
> final path component in the GUID.  Or something like a file extension:  
> .456
> 
> 
> As far as the server goes:
> 
> The server accesses the SCCS and caches the part by revision ID, say in
> a hashtable.  It responds to http protocol requests.  Because the
> server's commitment is to honor the same sexpr text for any GUID, it in
> theory never has to flush its cache.  New commits to the SCCS by the
> librarians go in as a newer version of a part and not actual
> replacement.  Like a webpage, and because of inclusion, you may have to
> hit various LIBRARY_SOURCEs multiple times get everything into your
> local LIBRARYs in RAM.
> 
> So for the remote case:
> 
> EESCHEMA -> LIBRARY -> LIBRARY_SOURCE -> http -> SERVER, say in python
> or java or apache using websvn derivative -> SCCS
> 
> An calming consideration in all this is that much of what we are dealing
> with here is read only, cached data, and it eventually ALL finds it way
> into one of the open LIBRARYs (caches).  So it is cached in our local
> LIBRARY and if remote, also in the server.  With a goofy query string
> the server could give you SVG instead of sexpr so you can actually hit
> it with a web browser also and see what you are looking at.
> 
> To do the SVG form, the server would have to in line expand all the
> inheritance, and this would require that the repo not reference anything
> outside itself.

This concept has interesting implications for the 3D models in the PCB
editor as well.

> 
>>   
>>> To test whether you get what I am saying, and fully understand the
>>> concept of a parts list, it should be noted that I did not list "parts
>>> list" as a "library source" for the schematic in which it resides.  A
>>> parts list role is very unique with respect to the schematic in which it
>>> resides.  It is the instantiation factory for that schematic and no
>>> others.  However it can act as a library source for other schematics. 
>>> This design will keep the parts list complete within every schematic,
>>> and every schematic will stand alone, now and forever.  :) But this
>>> assumes the inherited components' library sources remain accessible
>>> forever, and with a GUID that includes a version number, and with a
>>> stable remote library source, I think it could actually happen that way.
>>>     
>> If we could pull this off, that would be excellent.  One of the never ending
>> headaches with every CAD system I have ever used is libraries.  It would be
>> great if I could copy a project folder from one computer to another without
>> having to remember to copy all of the library dependencies to each computer.
>>
>>   
>>> The notion of loading needs to be better defined, since *deferring* may
>>> be desired in the remote case.  There should be a "get contents"
>>> (directory listing) function with a component specific load after that. 
>>> Otherwise the remote library sources could be too slow.  Some more
>>> thought needs to be put into this.
>>>     
>> The remote case is going to be the most difficult one to solve. 
> 
> Not by much.  Not if the APIs are designed properly up front, the
> abstraction makes the client code identical.
> 
> The server does caching too, and I think a good sized server would have
> all the components cached in RAM anyway.  It would be like hitting a
> website that has all the pages already cached in RAM.  And if we use
> http protocol for this, it is very do-able.
> 
> 
>>  I suppose you
>> could check to see if the local copy of the library repo is up to date with the
>> remote repo if remote access is available and update the local repo when
>> necessary.  You would always use the local copy of the repo when editing a
>> schematic to prevent the latency of accessing the remote repo.
>>   
> 
> For my personal use, I would not even feel compelled to copy the repo. 
> It is "out there" and ready, on the net.  A typical component would have
> what, less than 300 character's in the http content part of the reply?

My concern is when you do not have access to "out there".  There are
many reasons for this so it would be nice to have the option of keeping
a local copy to fall back on for those times when you don't have net access.

All in all it seems like a very sound concept.  It will be interesting
to see how it works in practice.

Wayne

> 
> 
> Dick
> 
> 
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
> 



Follow ups

References