kicad-developers team mailing list archive

[Dick Hollenbeck <dick@xxxxxxxxxxx>]

On 08/31/2013 08:23 AM, Brian Sidebotham wrote:
> Hi Guys,
> 
> I was going to send this direct to Dick, but I think this pickles through enough projects
> for it to be of use or concern for others too:
> 
> The Debug scripting builds are broken because something that includes the Python header
> that has _DEBUG defined alters a LOT of defines. Linking will fail with missing references
> all over the place where in a release build the functions boil down to nothing in order to
> speed the interpreter up.
> 
> (1) Python-a-mingw-us Debug build changes
> 
> The debug build of python-a-mingw-us was a little broken. Looking at how the standard
> Python build does a debug build means that all dynamically loaded extensions must have the
> debug suffix ("_d"). This is required because the import module searches for the import
> name with the debug suffix.
> 
> The standard build also adds the debug suffix to libpython and the main python exe.
> 
> I've changed the build to do this, and that works well.
> 
> The standard Python distribution doesn't really provide a debug build that you can easily
> use. I'd like Python-a-mingw-us to provide a developer-friendly debug build. We would also
> include "fixed" cmake modules for discovering the python install to use the correct libraries.

Then that's your baby.  I would never use it.  Maybe you will be the only person to ever
use it.  Certainly doubling the size of the a-mingw-us zip file is questionable.

If I had to debug something in the software stack on windows, I'd probably extract the
broken part and augment it with printf()s temporarily.  Sort like how you remove the
carburetor from the car or ATV before you rebuild it.  And this is not simply because I
don't have a build environment or debugger installed on Windows, but that is also a factor.



> 
> As far as I'm aware the reason for having the debug suffix on all "C" compiled parts of
> python is so that the debug and release versions can be pasted ontop of each other and
> share the same install path. Use python_d to use the debug enriched interpreter, and
> python for a speedy interpreter.

There are only two dynamic link layers affected here by python policy choices, yet there
can be more in the stack NOT necessarily affected.

1) pyhton.exe -> python.dll
2) python.dll -> extensions


Yet when we carve out openssl as a DLL, then you have

extension -> openssl dll


This is not any kind of standard, renaming a dll with a _d suffix.  Again, the whole debug
stack is for you apparently.  Other folks simply want it to work, and will use the release
build only.   When I did the regression testing, I had to do that on Windows.  I used
printf() to track down numerous problems, then removed those later after a-mingw-us passed
the test.  It was the release build being tested.

So the debug build is a yawner to me, and I am the second most likely person to use it.
Just imagine how unlikely the third most likely person is to use it.



> 
> (2) wxPython-cmake build changes
> 
> The only fixes required to the wxPython-cmake project are to include the fixed
> findpython*.cmake modules.
> 
> All dynamically loaded modules require the same python debug suffix for the debug version
> of the build.
> 
> Nothing else exciting.
> 
> (3) KiCad build changes
> 
> Again, the findpython*.cmake files require further fixing. Really nothing much tbh - at
> least to get a working build system.
> 
> The PCBNEW dynamically loaded module required the same python debug suffix in debug builds.
> 
> We also need to add the _DEBUIG preprocessor directive; But I expect this to change
> compilation on Linux and Mac.

Why?  Can't someone spell?

Why _DEBUG, we already have DEBUG as our KiCad standard?  Maybe we should submit another
patch and fix somebody's spelling.

I am resistant to adding this to KiCad's build scripts, at least wholesale.


> 
> Just getting the debug build working was enough to know that something was wrong with the
> scripting console initialisation as I instantly got a wxWidgets Debug Assert: AddPane()
> NULL window ptrs are not allowed...

Why the whole stack has to be built for debug is a mystery to me.  A person can run
release builds on top of debug libraries, this is simply a few changes in command line
options after all.

The asserts are not because you are running under a debugger, but because the asserts have
been compiled in as active.  This is one step away from my printf() alternative, its the
same concept.



> 
> It is strange - running through an IDE works okay, but running through gdb or via the
> command line results in the assertion. The error message from python says it cannot find
> _std_d.pyd - yet it is there and doesn't appear to have any dependency problems.

Further example of why printf() can make sense when a debugger does not, at least after
you have narrowed the search space sufficiently.


 So I'm
> not sure why this is failing yet. The clues as to what's up are clearly there, but I
> cannot piece them together yet.
> 
> So I had some questions because there are a few options for us.
> 
> (a) We/I can complete the fixes I've detailed above. I've selected the KiCad fix to only
> generate the debug python suffix on pcbnew.pyd on Windows if it is a debug build.  

Please explain why _DEBUG must go into KiCad build. This is not a current requirement and
I don't get it.  At least narrow it down to the subset of source files where you find it
solves a problem.  It should not get passed to all source files.  If we wrap python
interpreter instantiation into a C++ wrapper, then we can narrow down the files being
affected by such poor spelling.


> 
> (b) We can doctor Python-a-mingw-us to not append the debug suffix when it's built as a
> debug m
> oodule.


You are the only user of debug a-mingw-us builds, assume that.  If I had to debug anything
not windows specific, I'd simply compile it for Linux and step through it there.  There
source in some cases is the same.  If the bug was Windows specific, I'd add temporary
printf()s, and remove them 10 minutes later by bzr reverting.




> 
> Q:
> 
> (i) Does anyone see any output from the PyErr_Print() calls?

In what context?  Can we take the brain out of the patient and do the surgery on the
table?  Or are you talking about a nerve pathway leading back into the body?

In any context, outside KiCad, outside wxPython, are you saying PyErr_Print() simply does
not work?  Please explain the context.


 I think we need to extract
> the error information ourselves and push the output somewhere. Most likely we could do
> with a class called PyErr_PRINTER() or something so this can be called after we run
> internal python strings or files. I don't think we get any python error feedback at all at
> the moment.

Maybe there is a hook that you can install into the python interpreter.  I had wanted to
wrap the interpreter instantiation into a C++ class so we can:

a) hide 2.7.4 vs. 3.4 differences.

b) hide the complexity, so we can use the interpreter in lean contexts to evaluate python
expressions, not even full scripts.  We have REPORTER class now, maybe we can bundle the 3
ideas and get python to talk back to us.  Look for those hooks in python.dll.


> 
> (ii) What does Linux current do for debug builds? 

When I have to debug something under a debugger on Linux, I always compile it myself, so I
can fix the bug when I find it.  So I honestly don't know or care.  In 14 years of
developing on Linux, I have *never* used someone else's debug build of anything that I can
recall.  It is easier to simply build it myself.  And if building it in debug form is too
hard, I would add printf()s to the release build temporarily.


i.e. what python lib does it link to? I
> expect the python library detection is infact wrong. On a debug build it should detect the
> debug library. I've had to fix FindPythonLibs and FindPythonInterp to find suffixed
> versions on debug builds.
> 
> (iii) Does applying _DEBUG to Linux builds break them?
> 
> (iv) Is anyone else doing debug KiCad scripting builds on Windows?
> 
> KiCad-Winbuilder is nearly ready for release with scripting support, but I need to fix the
> console issue described above and to also fix Debug builds before I release.
> 
> Sorry this email's so long!!

If you want some help let me know where approximately the problem is by source file and
line number, at least the last known call.   We can figure it out.


Getting a sound set of C++ python interpreter instantiation wrappers is what I would do
first if I was on this.  Sometimes you may be interested in pure python, without wxWidgets
involved, other times wxWidgets use is intended.  Don't know if this needs to affect the
C++ wrapper class(es) or not.


Dick





> 
> Best Regards,
> 
> Brian.
> 
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
>