kicad-developers team mailing list archive

[Brian Sidebotham <brian.sidebotham@xxxxxxxxx>]

On 31 August 2013 18:51, Dick Hollenbeck <dick@xxxxxxxxxxx> wrote:

> On 08/31/2013 08:23 AM, Brian Sidebotham wrote:
> > Hi Guys,
> >
> > I was going to send this direct to Dick, but I think this pickles
> through enough projects
> > for it to be of use or concern for others too:
> >
> > The Debug scripting builds are broken because something that includes
> the Python header
> > that has _DEBUG defined alters a LOT of defines. Linking will fail with
> missing references
> > all over the place where in a release build the functions boil down to
> nothing in order to
> > speed the interpreter up.
> >
> > (1) Python-a-mingw-us Debug build changes
> >
> > The debug build of python-a-mingw-us was a little broken. Looking at how
> the standard
> > Python build does a debug build means that all dynamically loaded
> extensions must have the
> > debug suffix ("_d"). This is required because the import module searches
> for the import
> > name with the debug suffix.
> >
> > The standard build also adds the debug suffix to libpython and the main
> python exe.
> >
> > I've changed the build to do this, and that works well.
> >
> > The standard Python distribution doesn't really provide a debug build
> that you can easily
> > use. I'd like Python-a-mingw-us to provide a developer-friendly debug
> build. We would also
> > include "fixed" cmake modules for discovering the python install to use
> the correct libraries.
>
> Then that's your baby.  I would never use it.  Maybe you will be the only
> person to ever
> use it.  Certainly doubling the size of the a-mingw-us zip file is
> questionable.
>
> If I had to debug something in the software stack on windows, I'd probably
> extract the
> broken part and augment it with printf()s temporarily.  Sort like how you
> remove the
> carburetor from the car or ATV before you rebuild it.  And this is not
> simply because I
> don't have a build environment or debugger installed on Windows, but that
> is also a factor.
>
>
I prefer to have complete debug stacks for, well, debugging. But it's only
a user preference.


>
>
> >
> > As far as I'm aware the reason for having the debug suffix on all "C"
> compiled parts of
> > python is so that the debug and release versions can be pasted ontop of
> each other and
> > share the same install path. Use python_d to use the debug enriched
> interpreter, and
> > python for a speedy interpreter.
>
> There are only two dynamic link layers affected here by python policy
> choices, yet there
> can be more in the stack NOT necessarily affected.
>
> 1) pyhton.exe -> python.dll
> 2) python.dll -> extensions
>
>
> Yet when we carve out openssl as a DLL, then you have
>
> extension -> openssl dll
>
>
> This is not any kind of standard, renaming a dll with a _d suffix.  Again,
> the whole debug
> stack is for you apparently.  Other folks simply want it to work, and will
> use the release
> build only.   When I did the regression testing, I had to do that on
> Windows.  I used
> printf() to track down numerous problems, then removed those later after
> a-mingw-us passed
> the test.  It was the release build being tested.
>
> So the debug build is a yawner to me, and I am the second most likely
> person to use it.
> Just imagine how unlikely the third most likely person is to use it.
>
>
If all KiCad developers on Windows are happy without debug builds, then I
can keep these fixes local and it won't affect anyone.


>
> >
> > (2) wxPython-cmake build changes
> >
> > The only fixes required to the wxPython-cmake project are to include the
> fixed
> > findpython*.cmake modules.
> >
> > All dynamically loaded modules require the same python debug suffix for
> the debug version
> > of the build.
> >
> > Nothing else exciting.
> >
> > (3) KiCad build changes
> >
> > Again, the findpython*.cmake files require further fixing. Really
> nothing much tbh - at
> > least to get a working build system.
> >
> > The PCBNEW dynamically loaded module required the same python debug
> suffix in debug builds.
> >
> > We also need to add the _DEBUIG preprocessor directive; But I expect
> this to change
> > compilation on Linux and Mac.
>
> Why?  Can't someone spell?
>
> Why _DEBUG, we already have DEBUG as our KiCad standard?  Maybe we should
> submit another
> patch and fix somebody's spelling.
>
> I am resistant to adding this to KiCad's build scripts, at least wholesale.
>
>
> >
> > Just getting the debug build working was enough to know that something
> was wrong with the
> > scripting console initialisation as I instantly got a wxWidgets Debug
> Assert: AddPane()
> > NULL window ptrs are not allowed...
>
> Why the whole stack has to be built for debug is a mystery to me.  A
> person can run
> release builds on top of debug libraries, this is simply a few changes in
> command line
> options after all.
>
> The asserts are not because you are running under a debugger, but because
> the asserts have
> been compiled in as active.  This is one step away from my printf()
> alternative, its the
> same concept.
>



>
>

>
> >
> > It is strange - running through an IDE works okay, but running through
> gdb or via the
> > command line results in the assertion. The error message from python
> says it cannot find
> > _std_d.pyd - yet it is there and doesn't appear to have any dependency
> problems.
>
> Further example of why printf() can make sense when a debugger does not,
> at least after
> you have narrowed the search space sufficiently.
>
>
>  So I'm
> > not sure why this is failing yet. The clues as to what's up are clearly
> there, but I
> > cannot piece them together yet.
> >
> > So I had some questions because there are a few options for us.
> >
> > (a) We/I can complete the fixes I've detailed above. I've selected the
> KiCad fix to only
> > generate the debug python suffix on pcbnew.pyd on Windows if it is a
> debug build.
>
> Please explain why _DEBUG must go into KiCad build. This is not a current
> requirement and
> I don't get it.  At least narrow it down to the subset of source files
> where you find it
> solves a problem.  It should not get passed to all source files.  If we
> wrap python
> interpreter instantiation into a C++ wrapper, then we can narrow down the
> files being
> affected by such poor spelling.
>
>
*IF* a project is linked against a debug build of Python, _DEBUG must be
defined when compiling sources that include the python headers. Python
changes it's API a lot for debug builds, lots of macros in the release
version boil down to nothing whereas in the debug version they relate to
function calls.

However, if we are happy not to provide python source-level debug support
then everything should be fine by building the debug version of KiCad and
linking against the release versions of wxPython and libpython should be
fine.


> >
> > (b) We can doctor Python-a-mingw-us to not append the debug suffix when
> it's built as a
> > debug m
> > oodule.
>
>
> You are the only user of debug a-mingw-us builds, assume that.  If I had
> to debug anything
> not windows specific, I'd simply compile it for Linux and step through it
> there.  There
> source in some cases is the same.  If the bug was Windows specific, I'd
> add temporary
> printf()s, and remove them 10 minutes later by bzr reverting.
>
>
>
>
> >
> > Q:
> >
> > (i) Does anyone see any output from the PyErr_Print() calls?
>
> In what context?  Can we take the brain out of the patient and do the
> surgery on the
> table?  Or are you talking about a nerve pathway leading back into the
> body?
>
> In any context, outside KiCad, outside wxPython, are you saying
> PyErr_Print() simply does
> not work?  Please explain the context.
>
>
Sorry, the context is from the calls littered in
src/kicad/scripting/python_scripting.cpp where PyErr_Print() is used to
report python errors, except I don't see the output anywhere. Instead I had
to print them manually after fetching the Python error information.

I've used PyErr_Print in other projects, it definitely works.


>  I think we need to extract
> > the error information ourselves and push the output somewhere. Most
> likely we could do
> > with a class called PyErr_PRINTER() or something so this can be called
> after we run
> > internal python strings or files. I don't think we get any python error
> feedback at all at
> > the moment.
>
> Maybe there is a hook that you can install into the python interpreter.  I
> had wanted to
> wrap the interpreter instantiation into a C++ class so we can:
>
> a) hide 2.7.4 vs. 3.4 differences.
>
> b) hide the complexity, so we can use the interpreter in lean contexts to
> evaluate python
> expressions, not even full scripts.  We have REPORTER class now, maybe we
> can bundle the 3
> ideas and get python to talk back to us.  Look for those hooks in
> python.dll.
>
>
I will have a look a the REPORTER class, I've not seen it.


>
> >
> > (ii) What does Linux current do for debug builds?
>
> When I have to debug something under a debugger on Linux, I always compile
> it myself, so I
> can fix the bug when I find it.  So I honestly don't know or care.  In 14
> years of
> developing on Linux, I have *never* used someone else's debug build of
> anything that I can
> recall.  It is easier to simply build it myself.  And if building it in
> debug form is too
> hard, I would add printf()s to the release build temporarily.
>
>
I guess it sounds reasonable to build debug versions yourself. As I
mentioned above, everything should work fine linking against


>
> i.e. what python lib does it link to? I
> > expect the python library detection is infact wrong. On a debug build it
> should detect the
> > debug library. I've had to fix FindPythonLibs and FindPythonInterp to
> find suffixed
> > versions on debug builds.
> >
> > (iii) Does applying _DEBUG to Linux builds break them?
> >
> > (iv) Is anyone else doing debug KiCad scripting builds on Windows?
> >
> > KiCad-Winbuilder is nearly ready for release with scripting support, but
> I need to fix the
> > console issue described above and to also fix Debug builds before I
> release.
> >
> > Sorry this email's so long!!
>
> If you want some help let me know where approximately the problem is by
> source file and
> line number, at least the last known call.   We can figure it out.
>
>
> Getting a sound set of C++ python interpreter instantiation wrappers is
> what I would do
> first if I was on this.  Sometimes you may be interested in pure python,
> without wxWidgets
> involved, other times wxWidgets use is intended.  Don't know if this needs
> to affect the
> C++ wrapper class(es) or not.
>
>
Removing the debug build fixes from the list reduces the workload a lot. So
perhaps that's the way forward.

As for the scripting console, the failure ocurrs at
kicad\scripting\python_scripting.cpp:247. result here becomes false and the
error message from python is:

kicad\scripting\python_scripting.cpp(257): assert "Assert failure" failed
in CreatePythonShellWindow(): DLL load failed on file:
'kicad\bin\pylib\wx\_stc_d.pyd'. The specified module could not be found.

That assertion line is code I've added, it is the the result tested from
line 247 that fails.

After a few goes of debugging it now appears to *just work* - so please
don't waste any time on this. I don't know why it was/is failing, there has
literally been no change.

Building a debug build o KiCad and linking against release versions of
python and wxPython results in a lot of undefined references, which is why
I started down this debug build route. However, perhaps it's easier to find
why the KiCad debug build is not working. I'll look into it.

CMakeFiles\pcbnew.dir/objects.a(pcbnew_wrap.cxx.obj): In function
`SWIG_Python_AddErrorMsg':
D:/launchpad-dev/kicad-winbuilder-pyrelease/build/Debug/pcbnew/pcbnew_wrap.cxx:1020:
undefined reference to `_imp___Py_RefTotal'
D:/launchpad-dev/kicad-winbuilder-pyrelease/build/Debug/pcbnew/pcbnew_wrap.cxx:1020:
undefined reference to `_imp___Py_RefTotal'
D:/launchpad-dev/kicad-winbuilder-pyrelease/build/Debug/pcbnew/pcbnew_wrap.cxx:1024:
undefined reference to `_imp___Py_RefTotal'
D:/launchpad-dev/kicad-winbuilder-pyrelease/build/Debug/pcbnew/pcbnew_wrap.cxx:1024:
undefined reference to `_imp___Py_RefTotal'
D:/launchpad-dev/kicad-winbuilderCMakeFiles-pyrelease\_pcbnew.dir/build/objects.a/Debug(pcbnewPYTHON_wrap.cxx.obj/pcbnew):
In/pcbnew_wrap.cxx function:1024 `SWIG_Python_AddErrorMsg: undefined':
D reference:/launchpad to-dev `_imp___Py_NegativeRefcount/kicad'
D-winbuilder-pyrelease/build/Debug/pcbnew/scripting/pcbnewPYTHON_wrap.cxx:1020:
undefined reference to `_imp___Py_RefTotal'
D:/launchpad-dev/kicad-winbuilder-pyrelease/build/Debug:/launchpad/pcbnew-dev/scripting/kicad/pcbnewPYTHON_wrap.cxx-winbuilder:1020-pyrelease/build/Debug/pcbnew/pcbnew_wrap.cxx:1024:
undefined reference to `_imp___Py_Dealloc'
D:/launchpad-dev/kicad-winbuilder-pyrelease/build/Debug/pcbnew/pcbnew_wrap.cxx:1025:
undefined reference to `_imp___Py_RefTotal'
D:/launchpad-dev/kicad-winbuilder-pyrelease/build/Debug/pcbnew/pcbnew_wrap.cxx:1025:
undefined reference to `_imp___Py_RefTotal'
D:/launchpad: undefined-dev reference/kicad to-winbuilder
`_imp___Py_RefTotal-pyrelease/build/Debug/pcbnew/pcbnew_wrap.cxx:1025:
undefined reference to `_imp___Py_NegativeRefcount'
D:/launchpad-dev/kicad-winbuilder-pyrelease/build/Debug/pcbnew/pcbnew_wrap.cxx:1025:
undefined reference to `_imp___Py_Dealloc'
CMakeFiles\pcbnew.dir/objects.a(pcbnew_wrap.cxx.obj): In function
`SWIG_Python_SetErrorObj':
D:/launchpad-dev/kicad-winbuilder-pyrelease/build/Debug/pcbnew/pcbnew_wrap.cxx:1212:
undefined reference to `_imp___Py_RefTotal'
D:/launchpad-dev/kicad-winbuilder-pyrelease/build/Debug/pcbnew/pcbnew_wrap.cxx:1212:
undefined reference to `_imp___Py_RefTotal'
D:/launchpad-dev/kicad-winbuilder-pyrelease/build/Debug/pcbnew/pcbnew_wrap.cxx:1212:
undefined reference to `_imp___Py_NegativeRefcount'
D:/launchpad-dev/kicad-winbuilder-pyrelease/build/Debug/pcbnew/pcbnew_wrap.cxx:1212:
undefined reference to `_imp___Py_Dealloc'
CMakeFiles\pcbnew.dir/objects.a(pcbnew_wrap.cxx.obj): In function
`SWIG_Python_SetConstant':

etc.

Thanks for the reply - I think it's probably saved a lot of work!

Best Regards,

Brian.