kicad-developers team mailing list archive

[Jon Evans <jon@xxxxxxxxxxxxx>]

My idea is to make it possible for KiCad to talk to an external database.
The database itself (and its schema) will not be defined as part of this
spec, and will not be part of KiCad.

The only requirement is that you have some columns in your schema that
KiCad understands (for example, to point to the right symbols and
footprints in the KiCad libraries).
The planned interface to connect to the database is ODBC.  This would in
theory allow using sqlite to create a database as a file on disk, although
I anticipate that most users will be using something like MariaDB or
Postgres.

There has been some discussion of supporting talking to web interfaces via
some REST API, or even talking to arbitrary interfaces via Python
scripting, but that discussion should stay separate.
The original thread was about getting component information out of a
database and I just wanted to let people know that I am working on this.

People are welcome to also discuss the ideas of getting component
information from a web API or from some Python script.
But, I am not working on that right now, and there hasn't even been a
conversation started on what that spec would look like.

Best,
Jon

On Fri, Aug 28, 2020 at 1:42 PM Johann Wilhelm
<johann.wilhelm@wilhelm.consulting> wrote:

> Hi Jon,
>
> Well, you're idea was to define a database or at least a database schema
> if I understood this correctly:
>
> What I plan on implementing is not a "full" database management system, but
>
> rather an interface to grab info out of a database, just like you say.
>
> The only difference is, there are no plans to store actual symbols or
>
> footprints in the database.
>
> The symbols and footprints will still be stored in "normal" KiCad library
>
> files; the database will just contain "pointers" telling KiCad which symbol
>
> to use (and what library it can be found in).
>
>
>
> I was pointing toward a specification of a data format.So either read the
> data from a file or webinterface (that's why I used URI).
> My script-set currently gives me that information via
> http://localhost:8000/API/Component/getMatchWisdom
>
> Others would maybe like to save a file in ~/.KiCad (so the URI would be
> file://~/.KiCad)
>
> If "database"/"database interface" would include something which could
> read from files and/or web resources, well, never mind my comment :)
>
> With "global parameter" I mean something which could be part of a .pro
> file.
>
> Regards,
>     Johann
>
>
> Am Fr., 28. Aug. 2020 um 19:04 Uhr schrieb Jon Evans <jon@xxxxxxxxxxxxx>:
>
>> Hi Johann,
>>
>> I am not sure exactly what you are arguing against.  I don't see any
>> difference between what you are looking for and what is in my spec.
>> I am not sure what you mean by "global URI parameter" but the part picker
>> will be able to filter using any of the external data present.
>>
>> Best,
>> Jon
>>
>> On Fri, Aug 28, 2020 at 12:56 PM Johann Wilhelm
>> <johann.wilhelm@wilhelm.consulting> wrote:
>>
>>> Hi everyone!
>>>
>>> I'm new here in the mailing list but I'm currently building my own
>>> electronics business around KiCad which I use and love for years now.
>>>
>>> I spent the last weeks, trying to tie together procurement, PCB design,
>>> and fabrication in a single script-set.
>>> Just to have it mentioned - before getting self-employed, I worked for
>>> 10 years in the electronics industry in a huge enterprise.
>>> So I had some strict requirements for my tooling and I have some very
>>> strong opinions on how my perfect workflow should look like :)
>>>
>>> I'm very sorry but I need to say: PLEASE, don't just throw a part
>>> database at the community!
>>> Why? Because everyone has different approaches how to do procurement and
>>> inventory management!
>>> It's ok (and actually good!) if you try to come up with something but
>>> PLEASE, go for a defined filter interface!
>>>
>>> What I would suggest implementing  - actually, I have plans to implement
>>> it myself in mid-term - is a simple checkbox in the component dialog named
>>> "Apply Parts Filter" and a global URI parameter that provides the filter
>>> (or better: the source of "wisdom" used by the filter). Maybe another
>>> parameter defining the default for the checkbox would be nice as well...
>>>
>>> I think even a simple CSV with columns for Symbol, Footprint, and Value
>>> could provide sufficient information for effective filtering/adaption of
>>> the symbol-tree.
>>>
>>> A CSV like this:
>>>
>>>
>>>
>>> "Audio:TLV320AIC23BPW", "TLV320AIC23BPW", "Package_SO:TSSOP-28_4.4x9.7mm_P0.65mm"
>>> "Device:R_Small", "1k", "Resistor_SMD:R_0805_2012Metric"
>>> "Device:R*", "12k4", "Resistor_SMD:R_0805_2012Metric"
>>> "Device:R*", "12k4", "Resistor_SMD:R_1206_3216Metric"
>>> "Device:R*", "10R", "Resistor_SMD:R_0805_2012Metric"
>>>
>>>
>>> should result in a component tree like this:
>>>
>>> Audio
>>>
>>> TLV320AIC23BPW     <-- this symbol has a matching default footprint and
>>> value
>>>
>>> Device
>>>
>>> R  <-- this symbol is included in the CSV data
>>>
>>> 12k4     <-- a value of 12k4 would match Device:R and a corresponding
>>> symbol is added by the filter
>>>
>>> Resistor_SMD:R_0805_2012Metric   <--- both footprints would match
>>> Device:R 12k4 so the filter adds both symbols with the footprint and values
>>> fields complemented
>>> Resistor_SMD:R_0805_2012Metric   <--- ....
>>>
>>> 10R
>>>
>>> Resistor_SMD:R_0805_2012Metric
>>>
>>> R_Small
>>>
>>> 1k    <-- 1k only matches Device:R_Small
>>>
>>> Resistor_SMD:R_0805_2012Metric
>>>
>>> 12k4
>>>
>>> Resistor_SMD:R_0805_2012Metric
>>> Resistor_SMD:R_0805_2012Metric
>>>
>>> 10R
>>>
>>> Resistor_SMD:R_0805_2012Metric
>>>
>>>
>>> So the filter should add all symbols referenced in the CSV.
>>> If a symbol has no matching footprint and/or value => "create" them on
>>> fly
>>>
>>> With this, you can use the whole symbol library or the existing parts in
>>> your inventory.
>>> The best of both worlds!
>>>
>>> Regards,
>>>     Johann
>>>
>>>
>>> Am Fr., 28. Aug. 2020 um 16:38 Uhr schrieb Brian <lotharyx@xxxxxxxxx>:
>>>
>>>> Just want to add my +1 for the interface approach.  I am glad to hear
>>>> that is the intent (I’ve not had time to read the proposal).  With such an
>>>> interface, as has been pointed out already, data-source-specific
>>>> implementations should be relatively straightforward, and then I don’t have
>>>> to potentially throw away my years of privately curated data or figure out
>>>> how to cram it into some alternate schema.  All I would need to do is write
>>>> the code to answer the questions presented by the interface.
>>>>
>>>> Hopefully in the next few days I’ll be able to read Jon’s draft spec
>>>> and comment from a better-informed position.
>>>>
>>>> Cheers,
>>>> -Brian
>>>>
>>>> On Aug 28, 2020, at 9:43 AM, Jon Evans <jon@xxxxxxxxxxxxx> wrote:
>>>>
>>>> 
>>>> Hi Clemens,
>>>>
>>>> On Fri, Aug 28, 2020 at 9:34 AM Clemens Koller <cko@xxxxxxxxx> wrote:
>>>>
>>>>> Hello!
>>>>>
>>>>> This is related to the previous thread: "Automatic assignment of
>>>>> footprint with a database"
>>>>>
>>>>> I would generally prefer assemble real components on a real PCB right
>>>>> from the beginning instead of first placing generic components and then
>>>>> assign footprints + manufacturers + types + x manually. This seems extra
>>>>> work for each component which could possibly be avoided.
>>>>>
>>>>
>>>> Me too!
>>>>
>>>>
>>>>> So, regarding on the Kicad codebase, I would very likely not recomment
>>>>> to embed a full component management / database system, since this might
>>>>> vary from site to site and even from project/assembly house to
>>>>> project/house. But it would be great to be able to have an interface to
>>>>> grab a component out of a database and Kicad grabs all desired / a
>>>>> selection of individual columns out of the database as needed.
>>>>> This might include the actual footprints stored in the database as
>>>>> well.
>>>>>
>>>>
>>>> What I plan on implementing is not a "full" database management system,
>>>> but rather an interface to grab info out of a database, just like you say.
>>>> The only difference is, there are no plans to store actual symbols or
>>>> footprints in the database.
>>>> The symbols and footprints will still be stored in "normal" KiCad
>>>> library files; the database will just contain "pointers" telling KiCad
>>>> which symbol to use (and what library it can be found in).
>>>>
>>>> BTW, the example schema in your email looks very familiar to me.  This
>>>> is the kind of data source that can be used with the feature I am talking
>>>> about: just add columns to the schema for tracking which KiCad symbol and
>>>> footprint(s) should be used for a part.
>>>>
>>>> Regarding the advanced features you mention, some of them sound like
>>>> they would be handled by a PLM tool.
>>>> Some of the PLM tools I have worked with can interface to external
>>>> databases for managing this kind of component library for an EDA or
>>>> mechanical CAD tool.
>>>>
>>>> Best,
>>>> Jon
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~kicad-developers
>>>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~kicad-developers
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~kicad-developers
>>>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~kicad-developers
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~kicad-developers
>>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~kicad-developers
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>