kicad-developers team mailing list archive
-
kicad-developers team
-
Mailing list archive
-
Message #01339
Fileformats and library
Hi!
I would like to talk about the biggest issue of KiCAD.
I. The problem
--------------
After some experience[1] with KiCAD I think the biggest bottleneck in
KiCAD is the file formats of the footprints and schematics.
In the following section I will refer to footprints and schematics as
library (in place of
library and modules as located right now).
a) Location of the library
---------------------------
* In linux, all the libraries is located in /usr/share, and the user is
unable
to modify the schematic and the footprint. (It needs to copied somewhere
in the home directory).
b) Merging different libraries
------------------------------
* If I created a schematic, and saved into the appropriate library (ex:
device.lib), and
at work I create an another schematic. The merge of the two library is
really hard
(I need a text editor for this).
c) updates of the library
-------------------------
* Really hard to notice, if the default libraries (which comes with kicad)
has any new device inside.
d) using multiple locations(directories)
----------------------------------------
* To avoid unnecessary copies of the whole default library. It is necessary
to specify multiple
library location.
e) contribution back to the project
-----------------------------------
* It is really the same as the "merging different libraries" point. It makes
really hard to
contribute back to the project, because it is not a simple file, which I
could post back.
We need an unnecessary import/export which complicates life.
d) lack of versioning
------------------
* The title says it all. There are some case where would be nice to notify,
if a device was modified
and why. Currently in the default library there are many unconsistancies,
even If I correct
it. How someone would notice it? (ex: there is a footprint with soic,
while the other with s0ic.
So 0 vs o. hard to notice, but it is definitly an error. Other case: upper
case vs. lower case)
II. The solution
----------------
I was thinking many days now what would be the best for all the above
problems. I have also
read the mailing list for looking something similar. And I noticed, that
there is an ongoing
fileformat change effort. So I think the best time would be now to discuss
all of these
problems.
Informations about a device
------------------------
For a particular device, there are multiple information which needs to be
handle.
We need to handle: footprints, schematics, 3d representation, documentation
about the device
Let's take an example: tlp-281 optocoupler.
Which has a schematic (a led and a photosensitive transistor packaged
together),
has a footprints (smd, 4 legs, 1.27mm between two legs), and has a
phyisical package, which is now a soic4, which determines the 3d
representation.
The concrete model (tlp281) determine the datasheet and the documentation,
which is unique between all the devices.
Orthogonality
-------------
a) schematic:
-------------
* The schematic implies footprints (for example a transistor implies all the
footprints which
has three legs. Or there are maybe device, which comes just some packages).
b) footprint:
-------------
* A particular footprint implies a few physical package (3d representation).
c) package:
-----------
* package<-> 3d model is a 1:1 relationship.
One schematic/footprint/package one file:
-----------------------------------------
This is the BIGGEST issue about the current design. Really.
There is no need to collect multiple schematic inside one file. There are
dedicated tools for that (an archiver: .zip, .tar.gz), if the size matters.
There is a requirement about the new file format, that it needs to be
human readable and human (easily) understandable.
Have you managed to handle a file which has more then 2000 line of code?
So if the new fileformat needs to be human readable it needs also human
managable too.
Most important proposal:
------------------------
So I propose, that we split all the schematic bundles to invidual files.
Judging from the fileformat, it is *really* EASY.
It eliminates a bunch of problem. It makes really easy to contribute
back to the project.
Merging the documentation to appropriate file (future):
-------------------------------------------------------
Why handle separate the schematic and the documentation? (74xx.lib, 74xx.dcm
)
I think the best would be that all the invidual schematics contains
also his documentation too.
Proposed directory structure:
-----------------------------
Here is the directory structure what I think would be the best:
/library/
This is the root of all information (not sourcecode)
/library/schematic/
It contains all the schematics splitting to the appropriate subdirectories.
/library/schematic/audio/*.xxx
The current library/audio.lib file splitted to individual files.
/library/schematic/device/*.xxx
The current library/device.lib file splitted to individual files.
library/footprint/
It contains all the footprints separated to the appropriate folder.
library/footprint/display/*.xxx
The current modules/display.mod file splitted to individual files.
library/packages3D
Nothing needs to be done. The current structure is fine.
library/device
A new proposal here. It contains a file per physical device.
library/device/transistor/*.xxx
It contains all the transistors. A file then should contain information
about schematic, footprint, 3dpackage, and datasheet (url, or uri to a pdf).
Changes in the project file (.pro)
--------------------------------
Currently, when a project is created, the libraries (what the project are
using)
is stored inside the schematic (.sch), and inside the board design(.brd).
It is necessary, that all the information about the libraries
(what the user are using) stored at one location. And I think the best would
be inside the .pro file.
And while we are it, it would be nice also to use standard URI-s instead of
C:\\Document and settings\\Username\\Desktop\\kicad\\library and
/home/username/Desktop/kicad/library
becomes something like:
file:///c/document and settings/username/desktop/kicad/library and
file:///home/username/desktop/kicad/library
And would be necessary also to be able specify multiple library, something
like:
<schematic>
file:///c/program files/kicad/library/schematic/
file:///c/document and settings/username/desktop/kicad/library/schematic
file:///home/username/desktop/kicad/library/schematic
</schematic>
So the user moves their project between different os, kicad would find
always the appropriate libraries. And search inside multiple location
(so system libraries and user libraries too)
Future file format
------------------
I dont think Im the right person to even propose anything about the
future fileformat.
But I would like to point out, that if the schematics, footprints are
splitted
into invidual files, the future file format are more easy to implement, and
even the user can mix the new format and the old one! So the transition
could
be seamless and painless.
Implementation
--------------
I tried to propose the needed change in a way which are easy to implement
in small incemental steps.
Contribution and exporting a device
-----------------------------------
If somebody want to share their work with others or contribute back to the
project
the easiest way would be to have an export function from the program, which
would make the following actions:
* Collect the informations about a device
(device file (description of the device), footprint file, schematic file,
3dpackage file, datasheet)
* place the files into a temp directory with the device name, like bc547b
* zip the whole directory: bc547.zip
Versioning
----------
As all te device is handled as individual files, versioning is really easy.
Just put all the files into svn, and all the history of a particular package
are traced, and can diff between versions, contributors are traced too for
free.
So if there are any problem with a device (copyright infrigement or
anything),
to resolve an issue is simply remove the device, or revert the appropriate
change of the device.
About me
--------
Im just somebody who wants to use KiCAD, and toying with the idea to write a
package/schematic explorer in python and pygtk.
(or wxpython, if it turns out good, and the project wants to merge it)
Afterwords
----------
I hope that my words are not harsh (not my purpose). Im not criticizing the
project,
I just want to propose some ideas. If you read something inappropriate,
just ignore it (im not native english speaker).
All I want, is some brainstorming about the current situation.
Thank you really much for reading my proposal.
Best regards,
Khiraly
[1]:
Some personal experience: I have created a board (giant LED display), there
was two
device (SA40-19 100mm height led display, and tlp281, tlp281-4 optocoupler)
which
not existed in KiCAD. I have created the schematic of the optocoupler and
the footprint of the led display, and optocoupler.
Now I wanted modify my board, and all the footprints have been disappeared
from
the modules library. Thx there is in my .brd file too, so I will be able to
recover it using a text editor.
(I have finally recovered it, and I needed to add and remove my libraries in
cvpcb
several times along using text editor of course.)
Additional note: I use two computers (windowsXP at work, linux at home).
------=_Part_7599_9828712.1206899272947 Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
<span style="font-family: courier new,monospace;">Hi!</span><br style="font-family: courier new,monospace;"><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">I would like to talk about the biggest issue of KiCAD.</span><br style="font-family: courier new,monospace;">
<br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">I. The problem</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">--------------<br>
<br>After some experience[1] with KiCAD I think the biggest bottleneck in </span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">KiCAD is the file formats of the footprints and schematics. </span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">In the following section I will refer to footprints and schematics as library (in place of</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">library and modules as located right now).</span><br style="font-family: courier new,monospace;">
<br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">a) Location of the library</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">---------------------------</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">* In linux, all the libraries is located in /usr/share, and the user is unable<br> to modify the schematic and the footprint. (It needs to copied somewhere <br> in the home directory). <br>
</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">b) Merging different libraries</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">------------------------------</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">* If I created a schematic, and saved into the appropriate library (ex: device.lib), and </span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">at work I create an another schematic. The merge of the two library is really hard</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">(I need a text editor for this).</span><br style="font-family: courier new,monospace;"><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">c) updates of the library</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">-------------------------</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">* Really hard to notice, if the default libraries (which comes with kicad) has any new device inside.</span><br style="font-family: courier new,monospace;">
<br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">d) using multiple locations(directories)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">----------------------------------------</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">* To avoid unnecessary copies of the whole default library. It is necessary to specify multiple </span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> library location.</span><br style="font-family: courier new,monospace;">
<br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">e) contribution back to the project</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">-----------------------------------</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">* It is really the same as the "merging different libraries" point. It makes really hard to </span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> contribute back to the project, because it is not a simple file, which I could post back.</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> We need an unnecessary import/export which complicates life.</span><br style="font-family: courier new,monospace;"><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">d) lack of versioning</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">------------------</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">* The title says it all. There are some case where would be nice to notify, if a device was modified</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> and why. Currently in the default library there are many unconsistancies, even If I correct</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> it. How someone would notice it? (ex: there is a footprint with soic, while the other with s0ic.</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> So 0 vs o. hard to notice, but it is definitly an error. Other case: upper case vs. lower case)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"><br>
<br>II. The solution<br>----------------<br><br>I was thinking many days now what would be the best for all the above problems. I have also <br>read the mailing list for looking something similar. And I noticed, that there is an ongoing <br>
fileformat change effort. So I think the best time would be now to discuss all of these <br>problems.<br><br>Informations about a device<br>------------------------<br><br>For a particular device, there are multiple information which needs to be handle.<br>
We need to handle: footprints, schematics, 3d representation, documentation about the device<br><br>Let's take an example: tlp-281 optocoupler.<br>Which has a schematic (a led and a photosensitive transistor packaged together), <br>
has a footprints (smd, 4 legs, 1.27mm between two legs), and has a <br>phyisical package, which is now a soic4, which determines the 3d representation.<br>The concrete model (tlp281) determine the datasheet and the documentation, <br>
which is unique between all the devices.<br><br>Orthogonality<br>-------------<br><br>a) schematic:<br>-------------<br>* The schematic implies footprints (for example a transistor implies all the footprints which <br>has three legs. Or there are maybe device, which comes just some packages).<br>
<br>b) footprint:<br>-------------<br>* A particular footprint implies a few physical package (3d representation).<br><br>c) package:<br>-----------<br>* package<-> 3d model is a 1:1 relationship.<br><br><br>One schematic/footprint/package one file:<br>
-----------------------------------------<br><br>This is the BIGGEST issue about the current design. Really.<br><br>There is no need to collect multiple schematic inside one file. There are <br>dedicated tools for that (an archiver: .zip, .tar.gz), if the size matters.<br>
<br>There is a requirement about the new file format, that it needs to be <br>human readable and human (easily) understandable.<br><br>Have you managed to handle a file which has more then 2000 line of code?<br><br>So if the new fileformat needs to be human readable it needs also human<br>
managable too.<br><br>Most important proposal:<br>------------------------<br><br>So I propose, that we split all the schematic bundles to invidual files.<br>Judging from the fileformat, it is *really* EASY.<br><br>It eliminates a bunch of problem. It makes really easy to contribute <br>
back to the project.<br><br>Merging the documentation to appropriate file (future):<br>-------------------------------------------------------<br>Why handle separate the schematic and the documentation? (74xx.lib, 74xx.dcm)<br>
<br>I think the best would be that all the invidual schematics contains <br>also his documentation too.<br><br><br>Proposed directory structure:<br>-----------------------------<br><br>Here is the directory structure what I think would be the best:<br>
<br>/library/<br>This is the root of all information (not sourcecode)<br>
<br>/library/schematic/<br>It contains all the schematics splitting to the appropriate subdirectories.<br><br>/library/schematic/audio/*.xxx<br>The current library/audio.lib file splitted to individual files.<br><br>/library/schematic/device/*.xxx<br>
The current library/device.lib file splitted to individual files.<br><br>library/footprint/<br>It contains all the footprints separated to the appropriate folder.<br><br>library/footprint/display/*.xxx<br>The current modules/display.mod file splitted to individual files.<br>
<br>library/packages3D<br>Nothing needs to be done. The current structure is fine. <br><br>library/device<br>A new proposal here. It contains a file per physical device.<br><br>library/device/transistor/*.xxx<br>It contains all the transistors. A file then should contain information<br>
about schematic, footprint, 3dpackage, and datasheet (url, or uri to a pdf).<br><br>Changes in the project file (.pro)<br>--------------------------------<br><br>Currently, when a project is created, the libraries (what the project are using)<br>
is stored inside the schematic (.sch), and inside the board design(.brd).<br><br>It is necessary, that all the information about the libraries <br>(what the user are using) stored at one location. And I think the best would <br>
be inside the .pro file. <br><br>And while we are it, it would be nice also to use standard URI-s instead of <br>C:\\Document and settings\\Username\\Desktop\\kicad\\library and<br>/home/username/Desktop/kicad/library<br>
<br>becomes something like:<br>file:///c/document and settings/username/desktop/kicad/library and<br>file:///home/username/desktop/kicad/library<br><br>And would be necessary also to be able specify multiple library, something like:<br>
<schematic><br>file:///c/program files/kicad/library/schematic/<br>file:///c/document and settings/username/desktop/kicad/library/schematic<br>file:///home/username/desktop/kicad/library/schematic<br></schematic><br>
<br>So the user moves their project between different os, kicad would find<br>always the appropriate libraries. And search inside multiple location <br>(so system libraries and user libraries too)<br><br>Future file format<br>
------------------<br><br>I dont think Im the right person to even propose anything about the <br>future fileformat.<br><br>But I would like to point out, that if the schematics, footprints are splitted<br>into invidual files, the future file format are more easy to implement, and<br>
even the user can mix the new format and the old one! So the transition could<br>be seamless and painless.<br><br>Implementation<br>--------------<br><br></span><span style="font-family: courier new,monospace;">I tried to propose the needed change in a way which are </span><span style="font-family: courier new,monospace;">easy to implement <br>
in small incemental steps.<br><br>Contribution and exporting a device<br>-----------------------------------<br><br>If somebody want to share their work with others or contribute back to the project<br>the easiest way would be to have an export function from the program, which <br>
would make the following actions:<br>* Collect the informations about a device <br> (device file (description of the device), footprint file, schematic file, <br> 3dpackage file, datasheet)<br>* place the files into a temp directory with the device name, like bc547b<br>
* zip the whole directory: bc547.zip<br><br>Versioning<br>----------<br><br>As all te device is handled as individual files, versioning is really easy.<br>Just put all the files into svn, and all the history of a particular package <br>
are traced, and can diff between versions, contributors are traced too for free.<br><br>So if there are any problem with a device (copyright infrigement or anything), <br>to resolve an issue is simply remove the device, or revert the appropriate <br>
change of the device.<br><br>About me<br>--------<br><br>Im just somebody who wants to use KiCAD, and toying with the idea to write a <br>package/schematic explorer in python and pygtk. <br>(or wxpython, if it turns out good, and the project wants to merge it)<br>
<br><br>Afterwords<br>----------<br><br>I hope that my words are not harsh (not my purpose). Im not criticizing the project, <br>I just want to propose some ideas. If you read something inappropriate, <br>just ignore it (im not native english speaker).<br>
All I want, is some brainstorming about the current situation.<br>
<br><br>Thank you really much for reading my proposal.<br><br>Best regards, <br> Khiraly<br><br>[1]:</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">Some personal experience: I have created a board (giant LED display), there was two </span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">device (SA40-19 100mm height led display, and tlp281, tlp281-4 optocoupler) which </span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">not existed in KiCAD. I have created the schematic of the optocoupler and </span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">the footprint of the led display, and optocoupler.</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">Now I wanted modify my board, and all the footprints have been disappeared from</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">the modules library. Thx there is in my .brd file too, so I will be able to</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">recover it using a text editor. <br>(I have finally recovered it, and I needed to add and remove my libraries in cvpcb <br>several times along using text editor of course.)</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"><br>Additional note: I use two computers (windowsXP at work, linux at home). </span><br style="font-family: courier new,monospace;">
<br style="font-family: courier new,monospace;"><br style="font-family: courier new,monospace;">
------=_Part_7599_9828712.1206899272947--
Follow ups
