kicad-developers team mailing list archive
-
kicad-developers team
-
Mailing list archive
-
Message #34984
Re: Need guidance/hints on accessing list of available pcbnew footprints from python
Sorry, I meant wxEmptyString, not nullptr. The string argument tells the routine to enumerate only the one .pretty file; if you pass the empty string then it will enumerate all of the .pretty files in the fp-lib-table.
> On 14 Mar 2018, at 21:17, miles mccoo <mail@xxxxxxxxxx> wrote:
>
> from Jeff:
> wxArrayString footprintNames;
> fp_table->FootprintEnumerate( footprintNames, nullptr )
>
> will load footprintNames with all the footprints.
>
>
> That method doesn't take a nullptr. From fp_lib_table.h:
> void FootprintEnumerate( wxArrayString& aFootprintNames, const wxString& aNickname );
>
> Is there a function somewhere that will tell me all of the paths to libraries?
>
>
> But I do see now that the function hangs because I'm incorrectly calling the method. (though that doesn't justify the hang)
>
> in scripting window these commands assert complaining about an empty dir:
> import pcbnew
> pcbnew.FootprintEnumerate('')
>
>
> this hangs:
> import pcbnew
> pcbnew.FootprintEnumerate('MountingHole')
>
>
> This is the correct way to call it: (the python interface looks different, but it is, indeed, the same function[1]
> import pcbnew
> pcbnew.FootprintEnumerate('/home/mmccoo/kicad/kicad-footprints/MountingHole.pretty')
>
>
> [u'MountingHole_2.2mm_M2', u'MountingHole_2.2mm_M2_DIN965', u'MountingHole_2.2mm_M2_DIN965_Pad', u'MountingHole_2.2mm_M2_ISO14580', u'MountingHole_2.2mm_M2_ISO14580_Pad', u...
>
>
>
> Miles
>
>
>
> [1] from footprint.i
> wxArrayString footprintPyEnumerate( const wxString& aLibraryPath, bool aExitOnError )
> {
> wxArrayString footprintNames;
>
> if( aExitOnError )
> self->FootprintEnumerate( footprintNames, aLibraryPath );
> else
> {
> try
> {
> self->FootprintEnumerate( footprintNames, aLibraryPath );
> }
> catch( const IO_ERROR& error )
> {
> }
> }
>
> return footprintNames;
> }
>
> %pythoncode
> %{
> def FootprintEnumerate(self, libname):
> return self.footprintPyEnumerate( libname, True )
> %}
>
>
> On Mar 14, 2018 9:30 PM, "Jeff Young" <jeff@xxxxxxxxx <mailto:jeff@xxxxxxxxx>> wrote:
> Hi Miles,
>
> wxArrayString footprintNames;
> fp_table->FootprintEnumerate( footprintNames, nullptr )
>
> will load footprintNames with all the footprints.
>
> So in GDB you need to be looking at the size of aFootprintNames, not fp_table.
>
> Cheers,
> Jeff.
>
>
>> On 14 Mar 2018, at 20:01, miles mccoo <mail@xxxxxxxxxx <mailto:mail@xxxxxxxxxx>> wrote:
>>
>> Wayne said:
>> The footprint library table internally handles fallback library tables
>> so you can iterate over all of the same information available no matter
>> how deeply nested the fallback tables are from the project library table
>> without exposing the fallback table pointer. If you cannot access the
>> contents of the entire library table without exposing the fallback table
>> pointer, then the python bindings for the library table are broken.
>>
>>
>>
>> At this point, I'm not using python bindings. I'm in GDB poking around. I'm trying to figure out what bindings I need to add
>>
>> Again, I'm trying to get a list of all available footprints.
>>
>>
>>
>> If, in gdb, I step through this new (ie, in my sandbox only) function:
>> void GetFootprints2()
>> {
>> if( s_PcbEditFrame ) {
>> PROJECT *prj = &s_PcbEditFrame->Prj();
>> FP_LIB_TABLE* fp_table = prj->PcbFootprintLibs( s_PcbEditFrame->Kiway() );
>>
>> wxArrayString aFootprintNames;
>> fp_table->FootprintEnumerate(aFootprintNames, "MountingHole");
>> std::cout << "num enumerated " << aFootprintNames.size() << std::endl;
>> }
>> }
>>
>> I get this
>>
>> 187 fp_table->FootprintEnumerate(aFootprintNames, "MountingHole");
>> (gdb) print fp_table->GetCount()
>> $23 = 0
>> (gdb) print fp_table->IsEmpty()
>> Too few arguments in function call.
>> (gdb) print fp_table->IsEmpty(true)
>> $24 = false
>> (gdb) print fp_table->IsEmpty(false)
>> $25 = true
>> (gdb) print fp_table->fallBack->GetCount()
>> $27 = 92
>>
>>
>>
>> So, I can't iterate the rows via the At() method (which only looks at the current table)
>>
>>
>>
>> If I call FootprintEnumerate, my pcbnew hangs. both with empty string as last arg and as above. It seems to go into some other threads and I think it hangs in some sort of mutex.
>>
>> The stack trace is below. The other threads don't look any more interesting.
>>
>> Miles
>>
>>
>> (gdb) bt
>> #0 0x00007ffff60a1827 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x1dc1df0)
>> at ../sysdeps/unix/sysv/linux/futex-internal.h:205
>> #1 do_futex_wait (sem=sem@entry=0x1dc1df0, abstime=0x0) at sem_waitcommon.c:111
>> #2 0x00007ffff60a18d4 in __new_sem_wait_slow (sem=0x1dc1df0, abstime=0x0) at sem_waitcommon.c:181
>> #3 0x00007ffff60a197a in __new_sem_wait (sem=<optimized out>) at sem_wait.c:29
>> #4 0x00007fffda058fe8 in PyThread_acquire_lock () from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
>> #5 0x00007fffda02d926 in PyEval_RestoreThread () from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
>> #6 0x00007fffdaedc27c in pcbnewFinishPythonScripting () at /home/mmccoo/kicad/kicad/pcbnew/swig/python_scripting.cpp:270
>> #7 0x00007fffda86e3c7 in PCB::IFACE::OnKifaceEnd (this=0x7fffdbfe1ae0 <PCB::kiface>)
>> at /home/mmccoo/kicad/kicad/pcbnew/pcbnew.cpp:382
>> #8 0x000000000045d37d in KIWAY::OnKiwayEnd (this=0x6f3680 <Kiway>) at /home/mmccoo/kicad/kicad/common/kiway.cpp:489
>> #9 0x00000000004535f8 in PGM_SINGLE_TOP::OnPgmExit (this=0x6f3a40 <program>)
>> at /home/mmccoo/kicad/kicad/common/single_top.cpp:67
>> #10 0x0000000000454018 in APP_SINGLE_TOP::OnExit (this=0x756110) at /home/mmccoo/kicad/kicad/common/single_top.cpp:154
>> #11 0x00007ffff6c4af7f in CallOnExit::~CallOnExit (this=<synthetic pointer>, __in_chrg=<optimized out>)
>> at ../src/common/init.cpp:489
>> #12 wxEntry (argc=<optimized out>, argv=<optimized out>) at ../src/common/init.cpp:490
>> #13 0x000000000044f9ed in main (argc=1, argv=0x7fffffffd7c8) at /home/mmccoo/kicad/kicad/common/single_top.cpp:239
>>
>>
>> On Wed, Mar 14, 2018 at 6:05 PM, Wayne Stambaugh <stambaughw@xxxxxxxxx <mailto:stambaughw@xxxxxxxxx>> wrote:
>> On 3/14/2018 11:04 AM, miles mccoo wrote:
>> > Thanks for the tips, Orson. That gave me what I needed, sort of.
>> > Additional guidance needed.
>> >
>> >
>> >
>> > Let me describe what I have working and everyone can tell me if I
>> > understood everything correctly. You may want things moved around a bit
>> > as well.
>> >
>> > *high level view* is that I added pcbnew.GetFootprints(libname="") which
>> > returns a list of python dicts. like this:
>> > {'uniquepadcount': 1, 'padcount': 1, 'name':
>> > 'MountingHole_2.2mm_M2_DIN965_Pad', 'lib': 'MountingHole', 'doc':
>> > 'Mounting Hole 2.2mm, M2, DIN965', 'ordernum': 0, 'keywords': 'mounting
>> > hole 2.2mm m2 din965'}
>> >
>> >
>> > First, I'll mention that there already exists a footprint.i which has
>> > functions like FootprintLoad for getting an actual module instance.
>> > Couple issues there:
>> >
>> > * This is where I wanted to put GetFootprints, but *the code *(based
>> > off Orson's hint) *wants a Kiway, which I only know to get
>> > from PcbEditFrame*. pcbnew_scripting_helpers knows about
>> > the PcbEditFrame so that's where I put it.
>> > * footprints.i has a Footprint enumerate, but it wants a library path,
>> > which I don't have. More important, *it hangs when I run it*. I've
>> > CC'd JP Charras who authored that code (also one of the main Kicad
>> > people?).
>> >
>> >
>> > *Additional wierdness*
>> >
>> > The needed function should look like this:
>> > auto fp_info_list( FOOTPRINT_LIST::GetInstance( Kiway() ) );
>> > fp_info_list->ReadFootprintFiles( Prj().PcbFootprintLibs(), !nickname ?
>> > NULL : &nickname );
>> > for( auto& footprint : fp_info_list->GetList() ) {
>> >
>> > }
>> >
>> > fp_info_list is a local, which returns a list of unique footprint_info
>> > ptrs. The problems are:
>> >
>> > * footprint_info doesn't have a copy constructor. It's abstract
>> > virtual. So I can't make a copy of the GetList
>> > * I can't return GetList to SWIG since fp_info_list goes out of scope.
>> > * so I return fp_info_list to SWIG via a unique_ptr[1]
>> >
>> >
>> > I then have a swig typemap which gets GetList and generates the list of
>> > dicts. See [2] for the main code
>> >
>> >
>> > So that's what I have based off Orson's suggestion.
>> >
>> >
>> > *More detail on the other path I had been exploring* (which avoids some
>> > of the unique ptr stuff)*:*
>> >
>> > from pcb_edit_frame, I can get prj. from prj I can use FP_LIB_TABLE*
>> > PcbFootprintLibs. FP_LIB_TABLE is a LIB_TABLE which links to a fallback
>> > LIB_TABLE. That fallback is protected.*If I could expose it via a Get
>> > method, this path could work too*.
>>
>> The footprint library table internally handles fallback library tables
>> so you can iterate over all of the same information available no matter
>> how deeply nested the fallback tables are from the project library table
>> without exposing the fallback table pointer. If you cannot access the
>> contents of the entire library table without exposing the fallback table
>> pointer, then the python bindings for the library table are broken. If
>> you need to access the global library table, you can load it directly by
>> creating a new FP_LIB_TABLE object and using
>> FP_LIB_TABLE::GetGlobalTableFileName() but that should not be necessary.
>>
>> >
>> >
>> > the includefallback I was referring to is in lib_table_base.h
>> >
>> > /**
>> > * Return true if the table is empty.
>> > *
>> > * @param aIncludeFallback is used to determine if the fallback
>> > table should be
>> > * included in the test.
>> > *
>> > * @return true if the footprint library table is empty.
>> > */
>> > bool IsEmpty( bool *aIncludeFallback* = true );
>> >
>> >
>> > these APIs don't have that:
>> > int GetCount() { return rows.size(); }
>> >
>> > LIB_TABLE_ROW* At( int aIndex ) { return &rows[aIndex]; }
>> >
>> >
>> >
>> > A GetFallback method could do the trick? If I could get to it, I could
>> > iterate over
>> >
>> >
>> > Apologies for the long mail. *Getting the list of footprints seems
>> > harder than it should be? Anything I can do to help fix that? *Some
>> > refactoring, perhaps.
>> >
>> >
>> > Miles
>> >
>> >
>> > [1] swig doesn't play nicely with unique_ptrs.
>> > see http://www.swig.org/Doc3.0/SWIGDocumentation.html#SWIGPlus_nn19 <http://www.swig.org/Doc3.0/SWIGDocumentation.html#SWIGPlus_nn19>
>> > and https://stackoverflow.com/a/27699663/23630 <https://stackoverflow.com/a/27699663/23630>
>> >
>> > [2]
>> > // I am returing a shared_ptr to fp_info_list because that's the only way to
>> > // hold onto the list of footprint_infos long enough for the swig typename
>> > // to dictionary'ify it.
>> > // I can't just copy the footprint_infos; they as an abstract class
>> > (load()=0)
>> > // I don't like it, but this function should only be called by swig.
>> > std::unique_ptr<FOOTPRINT_LIST> GetFootprints(const wxString &libname)
>> > {
>> > // retval is a local unique ptr. If I just return GetList, it will
>> > be filled
>> > // with 0x0s. This is because when fp_info_list is desctructed, the
>> > unique_ptrs
>> > // returned by it will disappear as well.
>> >
>> > std::unique_ptr<FOOTPRINT_LIST> retval;
>> >
>> > if( s_PcbEditFrame ) {
>> > PROJECT *prj = &s_PcbEditFrame->Prj();
>> > retval = FOOTPRINT_LIST::GetInstance( s_PcbEditFrame->Kiway() );
>> > retval->ReadFootprintFiles( prj->PcbFootprintLibs(), !libname ?
>> > NULL : &libname );
>> > }
>> > return retval;
>> > }
>> >
>> >
>> >
>> >
>> >
>> > // http://www.swig.org/Doc3.0/SWIGDocumentation.html#SWIGPlus_nn19 <http://www.swig.org/Doc3.0/SWIGDocumentation.html#SWIGPlus_nn19>
>> > // theoretically, the following should suppress the use of SwigValueWrapper
>> > // but I couldn't get it to work:
>> > // %feature("novaluewrapper") std::unique_ptr<FOOTPRINT_LIST>;
>> >
>> > // The template stuff below is based on this answer:
>> > // https://stackoverflow.com/a/27699663/23630 <https://stackoverflow.com/a/27699663/23630>
>> > // I can't say I understand how/why it works.
>> > namespace std {
>> > %feature("novaluewrapper") unique_ptr;
>> > template <typename Type>
>> > struct unique_ptr {
>> > // we're not actually using the template feature of
>> > // swig to do anything. just want to avoid the use of SwigWrapper
>> > };
>> > }
>> > %template(UNIQUE_PTR_FOOTPRINT_LIST) std::unique_ptr<FOOTPRINT_LIST>;
>> >
>> >
>> > %typemap(out) std::unique_ptr<FOOTPRINT_LIST> {
>> >
>> > PyObject * retval = $result = PyList_New(0);
>> >
>> > if ( !$1 ) return retval;
>> >
>> > for( auto& footprint : $1->GetList() ) {
>> >
>> > PyObject *fpobj = PyDict_New();
>> > int fail = 0;
>> > fail |= PyDict_SetItemString(fpobj, "name",
>> > PyString_FromString(footprint->GetFootprintName()));
>> > fail |= PyDict_SetItemString(fpobj, "lib",
>> > PyString_FromString(footprint->GetNickname()));
>> > fail |= PyDict_SetItemString(fpobj, "doc",
>> > PyString_FromString(footprint->GetDoc()));
>> > fail |= PyDict_SetItemString(fpobj, "keywords",
>> > PyString_FromString(footprint->GetKeywords()));
>> >
>> > fail |= PyDict_SetItemString(fpobj, "padcount",
>> > PyInt_FromLong(footprint->GetPadCount()));
>> > fail |= PyDict_SetItemString(fpobj, "uniquepadcount",
>> > PyInt_FromLong(footprint->GetUniquePadCount()));
>> > fail |= PyDict_SetItemString(fpobj, "ordernum",
>> > PyInt_FromLong(footprint->GetOrderNum()));
>> >
>> > if (fail) {
>> > SWIG_exception_fail(SWIG_TypeError, "unable to convert
>> > FOOTPRINT_INFO list");
>> > }
>> > PyList_Append(retval, fpobj);
>> > }
>> >
>> > }
>> >
>> >
>> > On Tue, Mar 13, 2018 at 12:15 PM, Maciej Sumiński
>> > <maciej.suminski@xxxxxxx <mailto:maciej.suminski@xxxxxxx> <mailto:maciej.suminski@xxxxxxx <mailto:maciej.suminski@xxxxxxx>>> wrote:
>> >
>> > Hi Miles,
>> >
>> > Have you seen FOOTPRINT_VIEWER_FRAME::ReCreateFootprintList()
>> > (pcbnew/footprint_viewer_frame.cpp)? It might be the easiest way to go.
>> > Perhaps it could be wrapped in a function provided by the scripting
>> > interface.
>> >
>> > I could not find 'includefallback' option you have mentioned, would you
>> > point me to the relevant source code?
>> >
>> > Regards,
>> > Orson
>> >
>> > On 03/13/2018 10:49 AM, miles mccoo wrote:
>> > > In one of my python plugins, I want to know the list of available
>> > > footprints (mounting holes, in this case)
>> > >
>> > > Digging through the code, I can't make heads or tails of how to get this
>> > > information. There are a bunch of abstract class involved. impls....
>> > >
>> > > There are a couple possibilities, none of which seem clean.
>> > >
>> > > *Stuff from FOOTPRINT plugin*
>> > > looking in footprint.i, I see some APIs that are close.
>> > > If I have a directory path to a fp library, I can
>> > > call pcbnew.FootprintEnumerate for a nice list.
>> > >
>> > > but for that, I first have to have a list of fp library paths.
>> > *How do I
>> > > get such a list? *This seems like the closest answer. I see my
>> > config dir
>> > > has a fp-lib-table file which would be easy to parse. But that's a hack.
>> > >
>> > >
>> > >
>> > > *FOOTPRINT_LIST_IMPL possibility*
>> > > looking in load_select_footprint, I see it has a static FOOTPRINT_LIST_IMPL
>> > > which does have the APIs I'd need to get to a nice list of footprint_infos.
>> > > But it's static to that file. I could copy the relevant code to
>> > > python_scripting_helpers. but that feels messy.
>> > >
>> > > PCB_BASE_FRAME does have a method for popping up a table for a user to
>> > > choose a footprint, which is nice for UI. It could even be useful for
>> > > python plugins with some GUI stuff. But many python scripts will just want
>> > > a list of libraries and modules.
>> > >
>> > >
>> > >
>> > > *PROJECT possibility*
>> > > I tried exposing PROJECT to python as it has a number of useful sounding
>> > > API including PcbFootprintLibs which returns a FP_LIB_TABLE. By poking
>> > > around in gdb, I see that FP_LIB_TABLE has GetCount and IsEmpty. IsEmpty
>> > > returns false if I set includefallback to true. and the fallback list does
>> > > indeed have stuff in it. But it's protected and I don't see how to get to
>> > > it.
>> > >
>> > > Can I expose the fallback table via a public get method?
>> > >
>> > >
>> > > Is that a path that has any hope?
>> > >
>> > >
>> > >
>> > >
>> > > *So what's the most straight forward way to get to the list of
>> > libraries
>> > > and modules within them?*
>> > >
>> > > Thanks
>> > > Miles
>> > >
>> > >
>> > >
>> > > _______________________________________________
>> > > Mailing list: https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> > <https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>>
>> > > Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>> > <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>>
>> > > Unsubscribe : https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> > <https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>>
>> > > More help : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>
>> > <https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>>
>> > >
>> >
>> >
>> >
>> > _______________________________________________
>> > Mailing list: https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> > <https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>>
>> > Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>> > <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>>
>> > Unsubscribe : https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> > <https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>>
>> > More help : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>
>> > <https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>>
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > Mailing list: https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> > Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>> > Unsubscribe : https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> > More help : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>
>> >
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>> Unsubscribe : https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> More help : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>> Unsubscribe : https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> More help : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>
>
Follow ups
References
