kicad-developers team mailing list archive

[Bernhard Stegmaier <stegmaier@xxxxxxxxxxxxx>]

Hi again,

just wanted to share some of my OpenMP on macOS experience, FWIW:

I used both clang-5.0.1 and clang-6.0 from MacPorts.
KiCad builds fine with it.

It works fine on my build machine (b).
On my MacBook (a) such a build always crashes when ratsnest changes after I opened the PCB (e.g. move/update a footprint).
Initial ratsnest on opening the PCB is for whatever reason OK.
On a third machine (c) it crashes just like on (a).

There is some OpenMP stuff in the ratsnest code.
On my build machine I can stress the ratsnest code by moving a component around.
It is really using multiple threads according to CPU load.

That’s with clang-6.0.

A previous build with clang-5.0 doesn’t show this crash with ratsnest, but the 3d-viewer hang/crash problems I asked about in the other mail.
Again, 3d-viewer in OpenGL mode doesn’t work on my MacBook, but perfectly works on my build machine (same PCB, of course).

In backtraces of crashes always libomp task stuff is involved.

(b) and (c) are still on macOS 10.12, (a) is on 10.13.
It doesn’t seem to be about macOS version, because on one 10.12 machine the build is working, on the other not.

Of course, all machines are different HW. 
(b) is a 2012 i7. The other machines are a 2012 i5 and i3.
I even did a local build of libomp on my MacBook to rule out that it might use some i7 specific stuff that doesn’t run on i5/i3.
Didn’t help, I get the same crashes.

At the moment, I don’t really know what’s going on.
I looked at the code both for the 3d-viewer problem and the ratsnest stuff, but I don’t see anything being obviously wrong.
Of course, I don’t claim that I have understood every detail of it and I am no OpenMP expert.
It obviously works on a lot of linux boxes, so I guess it can’t be that wrong.

If someone wants to try the build:
  https://drive.google.com/open?id=1rpuKYxpHZTk5YiWe8s3LhIssHxJ9Y1oh <https://drive.google.com/open?id=1rpuKYxpHZTk5YiWe8s3LhIssHxJ9Y1oh>
I’d be happy to hear if it works or not, especially with OpenGL 3d-viewer and/or the rats nest use-case.


Regards,
Bernhard

> On 3. Mar 2018, at 20:33, Bernhard Stegmaier <stegmaier@xxxxxxxxxxxxx> wrote:
> 
> Hi all,
> 
> So, I played around a bit to get OpenMP and GCD into something like a “parallel_for” wrapper which either uses OpenMP or GCD.
> Unfortunately, OpenMP seems to require that the “for” statement directly follows the “#pragma omg for …”.
> That currently killed all my attempts to define such a “parallel_for” (as function, #define, ?) so that the code will stay the same for both.
> One option would be to pull the “#pragma omp for …” into the wrapper, but I guess this is not really acceptable (since you can’t specify any specific schedule parameters any longer).
> 
> I will think about it some more just because of the challenge… but I am not very optimistic at the moment to get something nice.
> If anyone else has a nice idea, I’d be very interested to learn freaky new stuff… :)
> 
> On the other side, I looked at using a non-Xcode OpenMP capable clang.
> The only caveat seems to be not to mix standard C++ libraries (in the past libc++/libstdc++ haven’t been compatible in some aspects - didn’t try to find out if that is resolved).
> Default on macOS now is libc++, but it seems that at least MacPorts builds clang with libstdc++ per default (no idea why).
> The MacPorts build can easily be switched to libc++.
> That clang-mp builds KiCad without problems even if dependencies have been built with Xcode clang.
> Everything seems to work as expected, multithreading is there where it should be.
> 
> So, for now it seems to be the most easy solution to switch to a suitable non-Xcode clang for building packages - if we want to have OpenMP for macOS.
> 
> AFAIK our build machines use Homebrew.
> Seems as if Homebrew decides which standard library to use depending on macOS version (https://docs.brew.sh/C++-Standard-Libraries <https://docs.brew.sh/C++-Standard-Libraries>).
> So, if everything is recent enough it might not be a problem at all… like it obviously also has been for Seth below.
> 
> 
> Regards,
> Bernhard
> 
>> On 1. Mar 2018, at 17:47, Seth Hillbrand <seth.hillbrand@xxxxxxxxx <mailto:seth.hillbrand@xxxxxxxxx>> wrote:
>> 
>> Hi All-
>> 
>> I was playing with this last night (don't normally compile on the mac) and found that using homebrew's llvm 3.8.1 seems to compile fine with -fopenmp.  Running some OMP test code from https://gist.github.com/sethhillbrand/45dfb50e5a1865ad65bda1d6522778c5 <https://gist.github.com/sethhillbrand/45dfb50e5a1865ad65bda1d6522778c5> shows expected result of 4 threads.  I didn't get OpenMP errors when compiling KiCad although I didn't really notice a substantial difference in speed for the quick tests I was running.  Maybe with more involved operations.
>> 
>> I'm not sure if this will require us to distribute a different libstdc++ with KiCad.  Probably.  Does that seem feasible to the packing team?
>> 
>> -S
>> 
>> 2018-03-01 7:23 GMT-08:00 Bernhard Stegmaier <stegmaier@xxxxxxxxxxxxx <mailto:stegmaier@xxxxxxxxxxxxx>>:
>> Hmm?
>> You keep everything as is (especially creating threads), but just put the “#pragma …” before the for-loop.
>> So, there is one thread for the progress and one for the workers.
>> In the workers thread OpenMP (if there) takes care of adding additional threads for parallelising the loop?
>> 
>> Or, am I wrong with that?
>> 
>> > On 1. Mar 2018, at 16:11, Jeff Young <jeff@xxxxxxxxx <mailto:jeff@xxxxxxxxx>> wrote:
>> >
>> > But then the progress reporter won’t work (and you’ve got no way to cancel).
>> >
>> > Non-pooling parallel threads are sufficient for zone filling, aren’t they?
>> >
>> >
>> >> On 1 Mar 2018, at 15:00, Bernhard Stegmaier <stegmaier@xxxxxxxxxxxxx <mailto:stegmaier@xxxxxxxxxxxxx>> wrote:
>> >>
>> >> For now it would probably be fine to just restore the pragma for the for loop optimisation.
>> >> Mac users are used to work single-threaded, all others would get back multithreading here.
>> >>
>> >>> On 1. Mar 2018, at 15:58, Tomasz Wlostowski <tomasz.wlostowski@xxxxxxx <mailto:tomasz.wlostowski@xxxxxxx>> wrote:
>> >>>
>> >>> On 01/03/18 15:43, Jeff Young wrote:
>> >>>> The purpose is it works on Mac.
>> >>>>
>> >>>> But it does appear I misread the std::max( omp_get_num_procs(), 2 ) part.
>> >>>>
>> >>>
>> >>> Thanks Jeff!
>> >>>
>> >>> Be aware that neither std::thread nor std::async have any concept of
>> >>> thread pooling - we need to look for a suitable library or write or own.
>> >>>
>> >>> Cheers,
>> >>> Tom
>> >>>
>> >>> _______________________________________________
>> >>> Mailing list: https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> >>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>> >>> Unsubscribe : https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> >>> More help   : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>
>> >>
>> >
>> 
>> 
>> _______________________________________________
>> Mailing list: https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>> Unsubscribe : https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> More help   : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>
>> 
>> _______________________________________________
>> Mailing list: https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>> Unsubscribe : https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>
>> More help   : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp