kicad-developers team mailing list archive

[Dick Hollenbeck <dick@xxxxxxxxxxx>]

On 10/05/2010 07:04 PM, Wayne Stambaugh wrote:
> On 10/5/2010 10:33 AM, Dick Hollenbeck wrote:
>   
>> I get paid to brainstorm, so brace yourself.
>>
>> Continuing with library brainstorming, the following is put on the table
>> for consideration and eventual evaluation. Brace yourself, this is out
>> of the box.  Open up your mind before proceeding.
>>
>> Remember the HTML editors which let you view HTML source and the
>> resultant GUI presentation at the same time?  One panel has the html
>> source, another shows you its effects.  The following concepts use that
>> UI paradigm.  Use the source Luke, use the source.
>>
>>
>> A library is not a library file:
>> -------------------------------
>>
>> The concept of a *library file* is gone.  There are none.  The concept
>> of a library can still exist but it is a RAM resident concept.  Each
>> component that we traditionally think of residing in a *library file* is
>> actually in its own file on disk in a directory (or repo or schematic). 
>> A library is only built in RAM during the loading of a collection of
>> components.
>>
>>
>> Loading components:
>> ------------------
>>
>> The act of loading components utilizes the notion of inheritance and
>> classical file inclusion like the C++ #include.  Let's go back an
>> understand inheritance thinking about Java. In a Java virtual machine
>> each class definition container has a method table.  That table is
>> initially populated with methods from the base class.  As the base class
>> is extended some of those methods are overloaded (replaced) with ones
>> from the derived class in the internal method table.  Additional fields
>> can be introduced in derived classes also, into what is a field table
>> for that class.  Now lets move that conceptual model to a component. 
>> Lets assume we have a pin table, and a graphics table.  We give a
>> component the ability to inherit from: 1) a base code fragment,  or 2)
>> symbol or 3) component.  In this case you can do pin additions, or pin
>> modifications, pin swapping, in the same way you can overload a Java
>> class's method table.  Pins can be overloaded or replaced, and graphical
>> primitives can be inherited and extended.
>>     
> Conceptually I really like this idea from a coding stand point.  If you
> are careful you could implement this so that the code maps one to one to
> your new conceptual model.  This would make it easier to implement new
> features will breaking the existing ones.  The only issue I see is how
> to convert existing libraries into the new format or do we abandon
> backwards capability and push on towards a superior implementation.
>   

I think changing the COMPONENT C++ object is wise.  We have to have a
few s-expression keywords that we would map onto a 1 for 1 C++ object:

1) (component ...)

for the pin table:

2) (pin ...)   # w/coordinates relative to the component's (x,y) origin.


for the graphics table:

3) (line ...)
4) (arc ...)
5) (circle ...)
6) (rect ...)

: yes more!

into a separate property table goes each property:

7) field or property



The rest of the keywords do not need to map onto "objects" but could map
onto "actions", and one that I can think of is "pin_del":

(pin_del 12)

Which would delete pin 12 should it already have been inherited.

So maybe you have "field_del"  or "property_del" also.  Trying to delete
graphics is more cumbersome.

We can and should write a conversion program for existing libraries to
new stuff.


class COMPONENT
{
    pins              // pin table

    properties        // aka fields collection

    graphics          // graphics primitives
}

Obviously keyword "pin" comes with a type which would imply the fixed
graphics for it.

more below:

>   
>> The magic happens during the loading phase of the S-expressions, so that
>> the "built up" component is fully assembled in RAM by using pieces from
>> various places "out there", while *loading*.
>>
>> If desired, the various pieces can be ear-marked with their origins in
>> full recognition of the existence of inheritance.  One might do this
>> only to facilitate graphical editing of a derived component, although if
>> the grammar is sweet enough, textual editing may be enough for all
>> editing of derived components.
>>     
> I have always preferred a well designed textual editing interface for
> creating drawings.  In many ways I find it easier than performing these
> operations graphically.  I however probably do not represent the larger
> group of users who prefer a GUI editor.  This may be one of those ideas
> to have our users group liaison check in with the users group to see how
> they feel about it.
>
>   
>> Re-loading is simply done by clicking a "parse" or "show" button.  The
>> the parser reloads the source stream and does file inclusions as called
>> for.  A portion of that source stream comes from the textual UI panel. 
>> Inherited portions come from a library (which is now a RAM resident
>> concept only).
>>
>>
>> Example
>> -------
>>
>> (component SOMENAME
>>   # inherit is like C++'s #include
>>   (inherit SOMEOTHERNAME WHERE)
>>   (pin 12 ... WHERE...ORIENTATION)
>>   (pin 13 ... WHERE...ORIENTATION)
>>   (pin 14 ...)
>> )
>>
>> Pin 12 and 13 are provided here and they are overloads because 12 and 13
>> have already been defined in SOMEOTHERNAME.  So this could be the means
>> of a *pin swap*.   However pin 14 is new, since SOMEOTHERNAME did not
>> have one, so this is an extension. In one example we see both a)
>> overloading and b) extension.
>>
>> This example would be a new SOMENAME component, and its graphics are
>> fully inherited from SOMEOTHERNAME, and can be extended here, but not
>> overloaded in any way that I can think up.
>>
>>
>> Where do they come from? and component versioning:
>> -------------------------------------------------
>> The ability to reference an external component by a GUID (globally
>> unique ID) of some kind lets you pull in components from anywhere in the
>> world.  A GUID inherently needs to respect versioning.  That is to say,
>> if you make an edit to a component that somebody else depends on, you
>> still have to provide the original version also.
>>     
> This is a great idea.  This prevents unwanted changes to your schematic
> when the underlying component is revised.  Doesn't Bazaar use the GUID
> concept for internally keeping track of revisions?  This would dovetail
> nicely into your GUID concept.  Where does this leave the concept of
> project libraries?  


As was suggested by Amir, we need to abstract the loading of libraries. 
A library is a RAM resident concept only, and should be thought of as a
cache.

Maybe we should use the term "library source" to contemplate what we now
think of as a "library file".

I see the need for four types of library sources:

1) a local library source which is simply a directory on the user's
disk, with separate component files in it.  This is one of two that are
writable.  The others are read only.

2) a local library source which comes from a SCCS repo, and is a repo
accessible via the local file system.  This is a read only library source.

3) any schematic with a "parts list" in it is a library source for a new
other schematic.  This is a read only source until you load that parts
list containing schematic purposefully to edit it.

4) a remote library source which comes from a global server and uses a
SCCS repo on the back end.  This is a read only library source.


Once these library sources are loaded into libraries in RAM, then their
distinctions are trivialized.  Because all access would be by library,
not library source.

If 1) is coded first, with a view to reserve a revision ID slot as part
of the GUID, then we could walk before we run, and the user could use
the library source as a "working directory" in a SCCS on his own,
without our knowledge, until we can run.  And during the migration
phase, I think this alludes to a 5) type of library source, namely the
read only form of 1) which is needed for the kicad supplied components
until they can be put behind a SCCS and become 4). 

The user would have to create his own 2) by "checking out" 4), and I
don't think we have to do that for him.  I for one would hope to have to
never do it, since I have decent Internet access.

The downside of coding 1) *first* is that you won't pay as much
attention to the caching needs.


> Do they become a local library repo in project
> directory or does all this component information get saved in the
> schematic file?
>   

If I get my parts list support (support here meaning user acceptance or
functionality) then all the components used in a schematic will *always*
reside in full in the parts list.  No exceptions.  The act of populating
a schematic specific parts list (well that is redundant since a parts
list by my definition is schematic specific), is a function of a good UI
and it would pull components in from various libraries.  Or symbols
which can be embellished in the parts list arena with s-expression
extension.  That parts list is available to other schematics, but only
as a library source, not as a parts list again, since a parts list must
live in the schematic that uses it.  In the parts list, there are
s-expressions defining what that part is, but inheritance can be used to
reference components in any library using a GUID.



>   
>> Enter the version control system.  Since each component and component
>> source code fragment exists in its own file, a source code control
>> system is used to house these files, and they can be accessed by using a
>> SCCS repo, file name and version number, over the Internet, or locally.
>>
>> One possibility for this is to build a tcp socket oriented bridge into
>> Bazaar's access API, which I assume lets you access any version of any
>> archived file.  Only one such server is needed for the "kicad supplied
>> libraries".  Behind the scenes the bridge/server simply uses launchad's
>> bazaar based repo.
>>
>> The loading of code fragments from afar need to be cached in RAM, so the
>> concept of a *library* does not go away.  A library is a RAM container
>> that caches those things coming from afar in order to enhance speed. 
>> But remember, kicad supplied libraries are essentially "read only" (for
>> all but those with the generating scripts).  So putting them under SCCS
>> and in launchpad as a primary place of dwelling is reasonable for those
>> with Internet access.  Others could check out the library branch.
>>     
> Does launchpad allow you to access repos without SSH keys? 

I said launchpad, but probably should have said bazaar, and I don't know
the answer to your question.

>  It may be a
> bit cumbersome for users to have to create a launchpad ID just to check
> out or keep up to date with the main repo.  

Indeed, and is a game breaker.  We only need read only access for the
kicad supplied components.  For remote access I think we need to keep
http: protocol on the table, and simply put a server up on a box
somewhere that hides the SCCS as a read only back end.  That can even be
coded in a language other than C++.  We only need one for the project,
but do not preclude 3rd party offerings.  We could make the server open
source so it can be re-used.  However if the  librarians don't put
barriers up, the kicad supplied remote library source can be modified
with the librariann's approval so multiple remote library sources are
not necessarily preferred.


> Maybe we could provide a
> guest user ID and SSH key to do this.
>   


>   
>>
>> How?
>> ----
>>
>> I can have this coded by Tuesday, given enough incentive.  (I'm just not
>> going to say which Tuesday.)
>>     
> Do I want to know what "enough incentive" entails?  Seriously though, it
> is a neat concept which you obviously have give a lot of thought to and
> I'm certainly interested in seeing how this would work.  Please keep me
> posted as I still have quite a bit of component library object clean up
> work to do.  If you have any questions about any of the component
> library object code or if there is anything I can do to help let me know.
>   

The loading and searching functions need to be identified and put into a
"LIBRARY_SOURCE" C++ class hierarchy to facilitate loading abstraction. 
This is a way to abstract the library loading, and remember most
versions of these loaders are read only, in fact most are.  For the
LIBRARYs that can be saved, we could do an end run around the
LIBRARY_SOURCE API, since the only case we have in my design is type 1)
and this is a simple matter of saving the s-expression file to a local
disk file.  Even type 3) is read only.  To edit 3) you have to load that
schematic with a view to edit it, then you can save the entire schematic
to disk with its parts list.

To test whether you get what I am saying, and fully understand the
concept of a parts list, it should be noted that I did not list "parts
list" as a "library source" for the schematic in which it resides.  A
parts list role is very unique with respect to the schematic in which it
resides.  It is the instantiation factory for that schematic and no
others.  However it can act as a library source for other schematics. 
This design will keep the parts list complete within every schematic,
and every schematic will stand alone, now and forever.  :) But this
assumes the inherited components' library sources remain accessible
forever, and with a GUID that includes a version number, and with a
stable remote library source, I think it could actually happen that way.

The notion of loading needs to be better defined, since *deferring* may
be desired in the remote case.  There should be a "get contents"
(directory listing) function with a component specific load after that. 
Otherwise the remote library sources could be too slow.  Some more
thought needs to be put into this.


Dick