← Back to team overview

kicad-developers team mailing list archive

Re: Fileformats and library

 

Dear Khiraly,

This is good stuff. I am surprised that you have not gotten anybody else to respond. I was waiting for others to comment before I did, but none have yet so I thought at least I would tell you that your ideas are appreciated, and seem to be quite well thought out.

Here is my understanding of your suggestions:


You suggest a two phase transition.

In the first phase, all we do is split the schematic symbols and footprints into individual files and teach eeschema and pcbnew how to scan a subdirectory tree (one or more such trees with different base directories would be nice) to find all the symbols and footprints. And commit each of these files to subversion.


In the second phase, and concurrent with support for first phase so a gradual migration can occur, you suggest moving to an integrated file per symbol, and that integrated file would have all the required pieces to describe a electronic component.


The first phase seems reasonable. I am not sure about the second phase. It seems you might be suggesting that for example all SMT 0805 footprints would point to (would be bundled with in phase II) the same 3d model? I think that train of thought needs more clarification and elaboration.

Otherwise this might be the beginnings of a nice improvement here. I hope others will now comment.


Dick



khiraly k wrote:
Hi!

I would like to talk about the biggest issue of KiCAD.

I. The problem
--------------

After some experience[1] with KiCAD I think the biggest bottleneck in
KiCAD is the file formats of the footprints and schematics.
In the following section I will refer to footprints and schematics as library (in place of
library and modules as located right now).

a) Location of the library
---------------------------
* In linux, all the libraries is located in /usr/share, and the user is unable to modify the schematic and the footprint. (It needs to copied somewhere
in the home directory).

b) Merging different libraries
------------------------------
* If I created a schematic, and saved into the appropriate library (ex: device.lib), and at work I create an another schematic. The merge of the two library is really hard
(I need a text editor for this).

c) updates of the library
-------------------------
* Really hard to notice, if the default libraries (which comes with kicad) has any new device inside.

d) using multiple locations(directories)
----------------------------------------
* To avoid unnecessary copies of the whole default library. It is necessary to specify multiple
library location.

e) contribution back to the project
-----------------------------------
* It is really the same as the "merging different libraries" point. It makes really hard to contribute back to the project, because it is not a simple file, which I could post back.
We need an unnecessary import/export which complicates life.

d) lack of versioning
------------------
* The title says it all. There are some case where would be nice to notify, if a device was modified and why. Currently in the default library there are many unconsistancies, even If I correct it. How someone would notice it? (ex: there is a footprint with soic, while the other with s0ic. So 0 vs o. hard to notice, but it is definitly an error. Other case: upper case vs. lower case)


II. The solution
----------------

I was thinking many days now what would be the best for all the above problems. I have also read the mailing list for looking something similar. And I noticed, that there is an ongoing fileformat change effort. So I think the best time would be now to discuss all of these
problems.

Informations about a device
------------------------

For a particular device, there are multiple information which needs to be handle. We need to handle: footprints, schematics, 3d representation, documentation about the device

Let's take an example: tlp-281 optocoupler.
Which has a schematic (a led and a photosensitive transistor packaged together),
has a footprints (smd, 4 legs, 1.27mm between two legs), and has a
phyisical package, which is now a soic4, which determines the 3d representation. The concrete model (tlp281) determine the datasheet and the documentation,
which is unique between all the devices.

Orthogonality
-------------

a) schematic:
-------------
* The schematic implies footprints (for example a transistor implies all the footprints which has three legs. Or there are maybe device, which comes just some packages).

b) footprint:
-------------
* A particular footprint implies a few physical package (3d representation).

c) package:
-----------
* package<-> 3d model is a 1:1 relationship.


One schematic/footprint/package one file:
-----------------------------------------

This is the BIGGEST issue about the current design. Really.

There is no need to collect multiple schematic inside one file. There are
dedicated tools for that (an archiver: .zip, .tar.gz), if the size matters.

There is a requirement about the new file format, that it needs to be
human readable and human (easily) understandable.

Have you managed to handle a file which has more then 2000 line of code?

So if the new fileformat needs to be human readable it needs also human
managable too.

Most important proposal:
------------------------

So I propose, that we split all the schematic bundles to invidual files.
Judging from the fileformat, it is *really* EASY.

It eliminates a bunch of problem. It makes really easy to contribute
back to the project.

Merging the documentation to appropriate file (future):
-------------------------------------------------------
Why handle separate the schematic and the documentation? (74xx.lib, 74xx.dcm)

I think the best would be that all the invidual schematics contains
also his documentation too.


Proposed directory structure:
-----------------------------

Here is the directory structure what I think would be the best:

/library/
This is the root of all information (not sourcecode)

/library/schematic/
It contains all the schematics splitting to the appropriate subdirectories.

/library/schematic/audio/*.xxx
The current library/audio.lib file splitted to individual files.

/library/schematic/device/*.xxx
The current library/device.lib file splitted to individual files.

library/footprint/
It contains all the footprints separated to the appropriate folder.

library/footprint/display/*.xxx
The current modules/display.mod file splitted to individual files.

library/packages3D
Nothing needs to be done. The current structure is fine.

library/device
A new proposal here. It contains a file per physical device.

library/device/transistor/*.xxx
It contains all the transistors. A file then should contain information
about schematic, footprint, 3dpackage, and datasheet (url, or uri to a pdf).

Changes in the project file (.pro)
--------------------------------

Currently, when a project is created, the libraries (what the project are using)
is stored inside the schematic (.sch), and inside the board design(.brd).

It is necessary, that all the information about the libraries
(what the user are using) stored at one location. And I think the best would
be inside the .pro file.

And while we are it, it would be nice also to use standard URI-s instead of
C:\\Document and settings\\Username\\Desktop\\kicad\\library and
/home/username/Desktop/kicad/library

becomes something like:
file:///c/document and settings/username/desktop/kicad/library and
file:///home/username/desktop/kicad/library

And would be necessary also to be able specify multiple library, something like:
<schematic>
file:///c/program files/kicad/library/schematic/
file:///c/document and settings/username/desktop/kicad/library/schematic
file:///home/username/desktop/kicad/library/schematic
</schematic>

So the user moves their project between different os, kicad would find
always the appropriate libraries. And search inside multiple location
(so system libraries and user libraries too)

Future file format
------------------

I dont think Im the right person to even propose anything about the
future fileformat.

But I would like to point out, that if the schematics, footprints are splitted into invidual files, the future file format are more easy to implement, and even the user can mix the new format and the old one! So the transition could
be seamless and painless.

Implementation
--------------

I tried to propose the needed change in a way which are easy to implement
in small incemental steps.

Contribution and exporting a device
-----------------------------------

If somebody want to share their work with others or contribute back to the project the easiest way would be to have an export function from the program, which
would make the following actions:
* Collect the informations about a device
(device file (description of the device), footprint file, schematic file,
3dpackage file, datasheet)
* place the files into a temp directory with the device name, like bc547b
* zip the whole directory: bc547.zip

Versioning
----------

As all te device is handled as individual files, versioning is really easy. Just put all the files into svn, and all the history of a particular package are traced, and can diff between versions, contributors are traced too for free.

So if there are any problem with a device (copyright infrigement or anything), to resolve an issue is simply remove the device, or revert the appropriate
change of the device.

About me
--------

Im just somebody who wants to use KiCAD, and toying with the idea to write a
package/schematic explorer in python and pygtk.
(or wxpython, if it turns out good, and the project wants to merge it)


Afterwords
----------

I hope that my words are not harsh (not my purpose). Im not criticizing the project,
I just want to propose some ideas. If you read something inappropriate,
just ignore it (im not native english speaker).
All I want, is some brainstorming about the current situation.


Thank you really much for reading my proposal.

Best regards,
Khiraly

[1]:
Some personal experience: I have created a board (giant LED display), there was two device (SA40-19 100mm height led display, and tlp281, tlp281-4 optocoupler) which
not existed in KiCAD. I have created the schematic of the optocoupler and
the footprint of the led display, and optocoupler.
Now I wanted modify my board, and all the footprints have been disappeared from the modules library. Thx there is in my .brd file too, so I will be able to
recover it using a text editor.
(I have finally recovered it, and I needed to add and remove my libraries in cvpcb
several times along using text editor of course.)

Additional note: I use two computers (windowsXP at work, linux at home).





--
Best Regards,

Dick


I pray we can elect a U.S. President that has the common sense to enforce our
existing immigration laws and stop the invasion from Mexico.







Follow ups

References