kicad-developers team mailing list archive
-
kicad-developers team
-
Mailing list archive
-
Message #44444
Re: Placing real components out of a database instead of: "Automatic assignment of footprint with a database"
https://calcite.apache.org/docs/adapter.html
Apache calcite is an SQL query engine for non-sql data sources.
On Sat, Aug 29, 2020, 21:21 Jeff Young <jeff@xxxxxxxxx> wrote:
> Surely there must be an open source impl of an ODBC interface on a CSV
> file?
>
> Although I’m not sure of the desire to avoid MySQL. It’s remarkably easy
> to set up an instance (or auto-deploy one with an app).
>
> Apologies if we’ve already talked about that; I’ll confess to not having
> followed this thread 100%….
>
> On 29 Aug 2020, at 14:11, Wayne Stambaugh <stambaughw@xxxxxxxxx> wrote:
>
> I would most likely reject any solution that was tied to a particular
> database. All this would do is open up a Pandora's box of complaints as
> to why we didn't use database A over database B. ODBC is the most
> flexible solution that I am aware of and allows users to choose their
> preferred database.
>
> Cheers,
>
> Wayne
>
> On 8/29/20 8:29 AM, Jon Evans wrote:
>
> Putting aside the fact that I think this feature isn't really aimed at
> hobbyists, I would not be opposed to people wanting to extend it beyond
> ODBC but that comes with extra complexity that must be handled.
>
> With ODBC, KiCad just needs to know about the interface to retrieve the
> data.
>
> With a CSV file, KiCad actually needs to read that file in and keep it
> in memory. Watch for modifications on disk, or else lock it
> exclusively. Things like that.
>
> I'm not sure I really see the advantage of a CSV-backed "database" over
> the existing KiCad library system, if we're talking about a single user.
>
> -Jon
>
> On Sat, Aug 29, 2020 at 8:19 AM Mark Roszko <mark.roszko@xxxxxxxxx
> <mailto:mark.roszko@xxxxxxxxx <mark.roszko@xxxxxxxxx>>> wrote:
>
> Sqlite was a quick&dirty way to test if my ideas would work.
>
> There are ODBC wrappers for SQLite........
>
> I mean libreoffice would do for the management.
>
> Yes, and you know what you would use? Not CSV files.
> LibreOffice Base / Microsoft Access.
> This is the office suite database that's basically SQLite and there
> are ODBC wrappers as well :D
>
> Also, why would a hobbyist fire up a sql database when a CSV file
>
> would be sufficient? I mean libreoffice would do for the management.
>
> KiCad's uses aren't limited to hobbyists...
> And, you assume there aren't hobbyists like me who will gladly take
> that ODBC link and spin up an frontend in a few hours to the whole
> system :D
>
>
> On Sat, Aug 29, 2020 at 4:22 AM Johann Wilhelm
> <johann.wilhelm@wilhelm.consulting> wrote:
>
> Hi there!
>
> Well, then my comment was not completely wrong.
>
> Sqlite was a quick&dirty way to test if my ideas would work. For
> my future productive system I really want to use a mix of
> couchdb and maybe postgres. i.e. a JSON document storage for the
> component information and sql for inventory management.
>
> So ODBC would not work well for me.
>
> Also, why would a hobbyist fire up a sql database when a CSV
> file would be sufficient? I mean libreoffice would do for the
> management.
>
> Additionally, I'd suggest looking at the BOM creation. There,
> external programs are called.
>
> So why not define a dataformat (xml, json, csv,...) and just
> call an external app or read from a file/uri?
>
> Anyways, I would volunteer for implementing some alternatives
> (read from file/uri and output of executable) to the ODBC
> interface if someone guides me through the KiCad procedures.
>
> Regards,
> Johann
>
>
> Jon Evans <jon@xxxxxxxxxxxxx <mailto:jon@xxxxxxxxxxxxx
> <jon@xxxxxxxxxxxxx>>> schrieb
> am Fr., 28. Aug. 2020, 19:54:
>
> My idea is to make it possible for KiCad to talk to an
> external database.
> The database itself (and its schema) will not be defined as
> part of this spec, and will not be part of KiCad.
>
> The only requirement is that you have some columns in your
> schema that KiCad understands (for example, to point to the
> right symbols and footprints in the KiCad libraries).
> The planned interface to connect to the database is ODBC.
> This would in theory allow using sqlite to create a database
> as a file on disk, although I anticipate that most users
> will be using something like MariaDB or Postgres.
>
> There has been some discussion of supporting talking to web
> interfaces via some REST API, or even talking to arbitrary
> interfaces via Python scripting, but that discussion should
> stay separate.
> The original thread was about getting component information
> out of a database and I just wanted to let people know that
> I am working on this.
>
> People are welcome to also discuss the ideas of getting
> component information from a web API or from some Python script.
> But, I am not working on that right now, and there hasn't
> even been a conversation started on what that spec would
> look like.
>
> Best,
> Jon
>
> On Fri, Aug 28, 2020 at 1:42 PM Johann Wilhelm
> <johann.wilhelm@wilhelm.consulting> wrote:
>
> Hi Jon,
>
> Well, you're idea was to define a database or at least a
> database schema if I understood this correctly:
>
> What I plan on implementing is not a "full" database
> management system, but
>
> rather an interface to grab info out of a database,
> just like you say.
>
> The only difference is, there are no plans to store
> actual symbols or
>
> footprints in the database.
>
> The symbols and footprints will still be stored in
> "normal" KiCad library
>
> files; the database will just contain "pointers"
> telling KiCad which symbol
>
> to use (and what library it can be found in).
>
>
>
> I was pointing toward a specification of a data
> format.So either read the data from a file or
> webinterface (that's why I used URI).
> My script-set currently gives me that information via
> http://localhost:8000/API/Component/getMatchWisdom
>
> Others would maybe like to save a file in ~/.KiCad (so
> the URI would be file://~/.KiCad)
>
> If "database"/"database interface" would include
> something which could read from files and/or web
> resources, well, never mind my comment :)
>
> With "global parameter" I mean something which could be
> part of a .pro file.
>
> Regards,
> Johann
>
>
> Am Fr., 28. Aug. 2020 um 19:04 Uhr schrieb Jon Evans
> <jon@xxxxxxxxxxxxx <mailto:jon@xxxxxxxxxxxxx
> <jon@xxxxxxxxxxxxx>>>:
>
> Hi Johann,
>
> I am not sure exactly what you are arguing against.
> I don't see any difference between what you are
> looking for and what is in my spec.
> I am not sure what you mean by "global URI
> parameter" but the part picker will be able to
> filter using any of the external data present.
>
> Best,
> Jon
>
> On Fri, Aug 28, 2020 at 12:56 PM Johann Wilhelm
> <johann.wilhelm@wilhelm.consulting> wrote:
>
> Hi everyone!
>
> I'm new here in the mailing list but I'm
> currently building my own electronics business
> around KiCad which I use and love for years now.
>
> I spent the last weeks, trying to tie together
> procurement, PCB design, and fabrication in a
> single script-set.
> Just to have it mentioned - before getting
> self-employed, I worked for 10 years in the
> electronics industry in a huge enterprise.
> So I had some strict requirements for my tooling
> and I have some very strong opinions on how my
> perfect workflow should look like :)
>
> I'm very sorry but I need to say: PLEASE, don't
> just throw a part database at the community!
> Why? Because everyone has different
> approaches how to do procurement and inventory
> management!
> It's ok (and actually good!) if you try to come
> up with something but PLEASE, go for a defined
> filter interface!
>
> What I would suggest implementing - actually, I
> have plans to implement it myself in mid-term -
> is a simple checkbox in the component dialog
> named "Apply Parts Filter" and a global URI
> parameter that provides the filter (or better:
> the source of "wisdom" used by the filter).
> Maybe another parameter defining the default for
> the checkbox would be nice as well...
>
> I think even a simple CSV with columns for
> Symbol, Footprint, and Value could provide
> sufficient information for effective
> filtering/adaption of the symbol-tree.
>
> A CSV like this:
>
>
>
> "Audio:TLV320AIC23BPW", "TLV320AIC23BPW", "Package_SO:TSSOP-28_4.4x9.7mm_P0.65mm"
>
> "Device:R_Small", "1k", "Resistor_SMD:R_0805_2012Metric"
> "Device:R*", "12k4",
> "Resistor_SMD:R_0805_2012Metric"
>
> "Device:R*", "12k4", "Resistor_SMD:R_1206_3216Metric"
>
> "Device:R*", "10R", "Resistor_SMD:R_0805_2012Metric"
>
>
> should result in a component tree like this:
>
> Audio
>
> TLV320AIC23BPW <-- this symbol has a
> matching default footprint and value
>
> Device
>
> R <-- this symbol is included in the
> CSV data
>
> 12k4 <-- a value of 12k4 would
> match Device:R and a corresponding
> symbol is added by the filter
>
> Resistor_SMD:R_0805_2012Metric
> <--- both footprints would
> match Device:R 12k4 so the
> filter adds both symbols with
> the footprint and values fields
> complemented
> Resistor_SMD:R_0805_2012Metric
> <--- ....
>
> 10R
>
> Resistor_SMD:R_0805_2012Metric
>
> R_Small
>
> 1k <-- 1k only matches Device:R_Small
>
> Resistor_SMD:R_0805_2012Metric
>
> 12k4
>
> Resistor_SMD:R_0805_2012Metric
> Resistor_SMD:R_0805_2012Metric
>
> 10R
>
> Resistor_SMD:R_0805_2012Metric
>
>
> So the filter should add all symbols referenced
> in the CSV.
> If a symbol has no matching footprint and/or
> value => "create" them on fly
>
> With this, you can use the whole symbol library
> or the existing parts in your inventory.
> The best of both worlds!
>
> Regards,
> Johann
>
>
> Am Fr., 28. Aug. 2020 um 16:38 Uhr schrieb Brian
> <lotharyx@xxxxxxxxx <mailto:lotharyx@xxxxxxxxx
> <lotharyx@xxxxxxxxx>>>:
>
> Just want to add my +1 for the interface
> approach. I am glad to hear that is the
> intent (I’ve not had time to read the
> proposal). With such an interface, as has
> been pointed out already,
> data-source-specific implementations should
> be relatively straightforward, and then I
> don’t have to potentially throw away my
> years of privately curated data or figure
> out how to cram it into some alternate
> schema. All I would need to do is write the
> code to answer the questions presented by
> the interface.
>
> Hopefully in the next few days I’ll be able
> to read Jon’s draft spec and comment from a
> better-informed position.
>
> Cheers,
> -Brian
>
> On Aug 28, 2020, at 9:43 AM, Jon Evans
> <jon@xxxxxxxxxxxxx
> <mailto:jon@xxxxxxxxxxxxx <jon@xxxxxxxxxxxxx>>>
> wrote:
>
>
> Hi Clemens,
>
> On Fri, Aug 28, 2020 at 9:34 AM Clemens
> Koller <cko@xxxxxxxxx
> <mailto:cko@xxxxxxxxx <cko@xxxxxxxxx>>> wrote:
>
> Hello!
>
> This is related to the previous
> thread: "Automatic assignment of
> footprint with a database"
>
> I would generally prefer assemble real
> components on a real PCB right from
> the beginning instead of first placing
> generic components and then assign
> footprints + manufacturers + types + x
> manually. This seems extra work for
> each component which could possibly be
> avoided.
>
>
> Me too!
>
>
> So, regarding on the Kicad codebase, I
> would very likely not recomment to
> embed a full component management /
> database system, since this might vary
> from site to site and even from
> project/assembly house to
> project/house. But it would be great
> to be able to have an interface to
> grab a component out of a database and
> Kicad grabs all desired / a selection
> of individual columns out of the
> database as needed.
> This might include the actual
> footprints stored in the database as well.
>
>
> What I plan on implementing is not a
> "full" database management system, but
> rather an interface to grab info out of a
> database, just like you say.
> The only difference is, there are no plans
> to store actual symbols or footprints in
> the database.
> The symbols and footprints will still be
> stored in "normal" KiCad library files;
> the database will just contain "pointers"
> telling KiCad which symbol to use (and
> what library it can be found in).
>
> BTW, the example schema in your email
> looks very familiar to me. This is the
> kind of data source that can be used with
> the feature I am talking about: just add
> columns to the schema for tracking which
> KiCad symbol and footprint(s) should be
> used for a part.
>
> Regarding the advanced features you
> mention, some of them sound like they
> would be handled by a PLM tool.
> Some of the PLM tools I have worked with
> can interface to external databases for
> managing this kind of component library
> for an EDA or mechanical CAD tool.
>
> Best,
> Jon
> _______________________________________________
> Mailing list:
> https://launchpad.net/~kicad-developers
> Post to :
> kicad-developers@xxxxxxxxxxxxxxxxxxx
> <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
> <kicad-developers@xxxxxxxxxxxxxxxxxxx>>
> Unsubscribe :
> https://launchpad.net/~kicad-developers
> More help :
> https://help.launchpad.net/ListHelp
>
> _______________________________________________
> Mailing list:
> https://launchpad.net/~kicad-developers
> Post to :
> kicad-developers@xxxxxxxxxxxxxxxxxxx
> <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
> <kicad-developers@xxxxxxxxxxxxxxxxxxx>>
> Unsubscribe :
> https://launchpad.net/~kicad-developers
> More help :
> https://help.launchpad.net/ListHelp
>
> _______________________________________________
> Mailing list:
> https://launchpad.net/~kicad-developers
> Post to :
> kicad-developers@xxxxxxxxxxxxxxxxxxx
> <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
> <kicad-developers@xxxxxxxxxxxxxxxxxxx>>
> Unsubscribe :
> https://launchpad.net/~kicad-developers
> More help : https://help.launchpad.net/ListHelp
>
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx
> <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
> <kicad-developers@xxxxxxxxxxxxxxxxxxx>>
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help : https://help.launchpad.net/ListHelp
>
>
>
> --
> Mark
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help : https://help.launchpad.net/ListHelp
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help : https://help.launchpad.net/ListHelp
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help : https://help.launchpad.net/ListHelp
>
Follow ups
References
-
Re: Placing real components out of a database instead of: "Automatic assignment of footprint with a database"
From: Jon Evans, 2020-08-28
-
Re: Placing real components out of a database instead of: "Automatic assignment of footprint with a database"
From: Brian, 2020-08-28
-
Re: Placing real components out of a database instead of: "Automatic assignment of footprint with a database"
From: Johann Wilhelm, 2020-08-28
-
Re: Placing real components out of a database instead of: "Automatic assignment of footprint with a database"
From: Jon Evans, 2020-08-28
-
Re: Placing real components out of a database instead of: "Automatic assignment of footprint with a database"
From: Johann Wilhelm, 2020-08-28
-
Re: Placing real components out of a database instead of: "Automatic assignment of footprint with a database"
From: Jon Evans, 2020-08-28
-
Re: Placing real components out of a database instead of: "Automatic assignment of footprint with a database"
From: Johann Wilhelm, 2020-08-29
-
Re: Placing real components out of a database instead of: "Automatic assignment of footprint with a database"
From: Mark Roszko, 2020-08-29
-
Re: Placing real components out of a database instead of: "Automatic assignment of footprint with a database"
From: Jon Evans, 2020-08-29
-
Re: Placing real components out of a database instead of: "Automatic assignment of footprint with a database"
From: Wayne Stambaugh, 2020-08-29
-
Re: Placing real components out of a database instead of: "Automatic assignment of footprint with a database"
From: Jeff Young, 2020-08-29
