kicad-developers team mailing list archive

[Bernhard Stegmaier <stegmaier@xxxxxxxxxxxxx>]

Orson,

no, the raytracing renderer is completely fine in that use-cases.

It is only the OpenGL renderer.
I first noticed the hang when I switched back from Raytracing to OpenGL (obviously after I did one of the 2 use-cases).

If it crashes, I get a backtrace of the crashing thread as follows
<<<
Thread 3 Crashed:
0   libomp.dylib                  	0x0000000109c728e2 __kmp_release_queuing_lock + 165
1   _pcbnew.kiface                	0x0000000111a1aaa3 CLAYER_TRIANGLES::AddToMiddleContourns(std::__1::vector<glm::tvec2<float, (glm::precision)0>, std::__1::allocator<glm::tvec2<float, (glm::precision)0> > > const&, float, float, bool) + 1011
2   _pcbnew.kiface                	0x0000000111a1ad52 CLAYER_TRIANGLES::AddToMiddleContourns(SHAPE_LINE_CHAIN const&, float, float, double, bool) + 546
3   _pcbnew.kiface                	0x0000000111a1b228 .omp_outlined. + 248
4   libomp.dylib                  	0x0000000109c81b93 __kmp_invoke_microtask + 147
5   libomp.dylib                  	0x0000000109c587b7 __kmp_invoke_task_func + 156
6   libomp.dylib                  	0x0000000109c57e88 __kmp_launch_thread + 267
7   libomp.dylib                  	0x0000000109c79e6f __kmp_launch_worker(void*) + 588
8   libsystem_pthread.dylib       	0x00007fff6735b6c1 _pthread_body + 340
9   libsystem_pthread.dylib       	0x00007fff6735b56d _pthread_start + 377
10  libsystem_pthread.dylib       	0x00007fff6735ac5d thread_start + 13
>>>

The main thread looks similar.
<<<
Thread 0:: Dispatch queue: com.apple.main-thread
0   libomp.dylib                  	0x0000000109c728e2 __kmp_release_queuing_lock + 165
1   _pcbnew.kiface                	0x0000000111a1aaa3 CLAYER_TRIANGLES::AddToMiddleContourns(std::__1::vector<glm::tvec2<float, (glm::precision)0>, std::__1::allocator<glm::tvec2<float, (glm::precision)0> > > const&, float, float, bool) + 1011
2   _pcbnew.kiface                	0x0000000111a1ad52 CLAYER_TRIANGLES::AddToMiddleContourns(SHAPE_LINE_CHAIN const&, float, float, double, bool) + 546
3   _pcbnew.kiface                	0x0000000111a1b228 .omp_outlined. + 248
4   libomp.dylib                  	0x0000000109c81b93 __kmp_invoke_microtask + 147
5   libomp.dylib                  	0x0000000109c587b7 __kmp_invoke_task_func + 156
6   libomp.dylib                  	0x0000000109c557dc __kmp_fork_call + 4682
7   libomp.dylib                  	0x0000000109c4d1fb __kmpc_fork_call + 183
8   _pcbnew.kiface                	0x0000000111a1b114 CLAYER_TRIANGLES::AddToMiddleContourns(SHAPE_POLY_SET const&, float, float, double, bool) + 628
9   _pcbnew.kiface                	0x0000000111a126ae C3D_RENDER_OGL_LEGACY::generate_holes_display_list(std::__1::list<COBJECT2D*, std::__1::allocator<COBJECT2D*> > const&, SHAPE_POLY_SET const&, float, float, bool) + 462
10  _pcbnew.kiface                	0x0000000111a12d39 C3D_RENDER_OGL_LEGACY::reload(REPORTER*) + 1545
11  _pcbnew.kiface                	0x0000000111a16e02 C3D_RENDER_OGL_LEGACY::Redraw(bool, REPORTER*) + 306
12  _pcbnew.kiface                	0x0000000111a09124 EDA_3D_CANVAS::OnPaint(wxPaintEvent&) + 628
>>>

I am not really familiar with OpenMP, but the AddToMiddleContourns methods look good in terms of synchronization (well, at first glance).


Regards,
Bernhard

> On 4. Mar 2018, at 20:06, Maciej Suminski <maciej.suminski@xxxxxxx> wrote:
> 
> Bernhard,
> 
> I suppose this is about raytrace rendering? Anyway, I see it crashing
> even without any design loaded. 3D viewer passes the first phase so I
> see the design rendered, but during 'Post processing shader' stage it
> crashes.
> 
> Cheers,
> Orson
> 
> On 03/04/2018 07:36 PM, Bernhard Stegmaier wrote:
>> Hi all,
>> 
>> could please anybody test the following on a Windows/Linux OpenMP version?
>> 
>> I have a PCB with only components placed (via “Update from Schematic”), no routing done yet.
>> Make sure 3d-viewer OpenGL rendering is OK, close 3d-viewer.
>> Now, edit a footprint (“e”) and do a “Update Footprint from Library” for only this footprint (no changes in footprint needed).
>> Open 3d-viewer again.
>> 
>> Or:
>> Open PCB as above with only components and no routing yet.
>> Make sure that 3d-viewer in OpenGL mode is fine, close 3d-viewer.
>> Delete some of the components and reimport them vie “Update PCB from Schematic”.
>> Open 3d-viewer again.
>> 
>> In both cases my OpenMP builds (clang-5.0.1 with libomp-3.1) crash or hang (all cores at 100%, grey window, nothing happens) 3d-viewer .
>> It doesn’t hang/crash in my non-OpenMP builds.
>> 
>> If I save the PCB after the changes, close and reopen KiCad, then 3d-viewer of OpenMP build is also fine again.
>> 
>> 
>> Regards,
>> Bernhard
>> 
>>> On 3. Mar 2018, at 20:33, Bernhard Stegmaier <stegmaier@xxxxxxxxxxxxx> wrote:
>>> 
>>> Hi all,
>>> 
>>> So, I played around a bit to get OpenMP and GCD into something like a “parallel_for” wrapper which either uses OpenMP or GCD.
>>> Unfortunately, OpenMP seems to require that the “for” statement directly follows the “#pragma omg for …”.
>>> That currently killed all my attempts to define such a “parallel_for” (as function, #define, ?) so that the code will stay the same for both.
>>> One option would be to pull the “#pragma omp for …” into the wrapper, but I guess this is not really acceptable (since you can’t specify any specific schedule parameters any longer).
>>> 
>>> I will think about it some more just because of the challenge… but I am not very optimistic at the moment to get something nice.
>>> If anyone else has a nice idea, I’d be very interested to learn freaky new stuff… :)
>>> 
>>> On the other side, I looked at using a non-Xcode OpenMP capable clang.
>>> The only caveat seems to be not to mix standard C++ libraries (in the past libc++/libstdc++ haven’t been compatible in some aspects - didn’t try to find out if that is resolved).
>>> Default on macOS now is libc++, but it seems that at least MacPorts builds clang with libstdc++ per default (no idea why).
>>> The MacPorts build can easily be switched to libc++.
>>> That clang-mp builds KiCad without problems even if dependencies have been built with Xcode clang.
>>> Everything seems to work as expected, multithreading is there where it should be.
>>> 
>>> So, for now it seems to be the most easy solution to switch to a suitable non-Xcode clang for building packages - if we want to have OpenMP for macOS.
>>> 
>>> AFAIK our build machines use Homebrew.
>>> Seems as if Homebrew decides which standard library to use depending on macOS version (https://docs.brew.sh/C++-Standard-Libraries <https://docs.brew.sh/C++-Standard-Libraries> <https://docs.brew.sh/C++-Standard-Libraries <https://docs.brew.sh/C++-Standard-Libraries>>).
>>> So, if everything is recent enough it might not be a problem at all… like it obviously also has been for Seth below.
>>> 
>>> 
>>> Regards,
>>> Bernhard
>>> 
>>>> On 1. Mar 2018, at 17:47, Seth Hillbrand <seth.hillbrand@xxxxxxxxx <mailto:seth.hillbrand@xxxxxxxxx> <mailto:seth.hillbrand@xxxxxxxxx <mailto:seth.hillbrand@xxxxxxxxx>>> wrote:
>>>> 
>>>> Hi All-
>>>> 
>>>> I was playing with this last night (don't normally compile on the mac) and found that using homebrew's llvm 3.8.1 seems to compile fine with -fopenmp.  Running some OMP test code from https://gist.github.com/sethhillbrand/45dfb50e5a1865ad65bda1d6522778c5 <https://gist.github.com/sethhillbrand/45dfb50e5a1865ad65bda1d6522778c5><https://gist.github.com/sethhillbrand/45dfb50e5a1865ad65bda1d6522778c5 <https://gist.github.com/sethhillbrand/45dfb50e5a1865ad65bda1d6522778c5>> shows expected result of 4 threads.  I didn't get OpenMP errors when compiling KiCad although I didn't really notice a substantial difference in speed for the quick tests I was running.  Maybe with more involved operations.
>>>> 
>>>> I'm not sure if this will require us to distribute a different libstdc++ with KiCad.  Probably.  Does that seem feasible to the packing team?
>>>> 
>>>> -S
>>>> 
>>>> 2018-03-01 7:23 GMT-08:00 Bernhard Stegmaier <stegmaier@xxxxxxxxxxxxx <mailto:stegmaier@xxxxxxxxxxxxx> <mailto:stegmaier@xxxxxxxxxxxxx <mailto:stegmaier@xxxxxxxxxxxxx>>>:
>>>> Hmm?
>>>> You keep everything as is (especially creating threads), but just put the “#pragma …” before the for-loop.
>>>> So, there is one thread for the progress and one for the workers.
>>>> In the workers thread OpenMP (if there) takes care of adding additional threads for parallelising the loop?
>>>> 
>>>> Or, am I wrong with that?
>>>> 
>>>>> On 1. Mar 2018, at 16:11, Jeff Young <jeff@xxxxxxxxx <mailto:jeff@xxxxxxxxx> <mailto:jeff@xxxxxxxxx <mailto:jeff@xxxxxxxxx>>> wrote:
>>>>> 
>>>>> But then the progress reporter won’t work (and you’ve got no way to cancel).
>>>>> 
>>>>> Non-pooling parallel threads are sufficient for zone filling, aren’t they?
>>>>> 
>>>>> 
>>>>>> On 1 Mar 2018, at 15:00, Bernhard Stegmaier <stegmaier@xxxxxxxxxxxxx <mailto:stegmaier@xxxxxxxxxxxxx> <mailto:stegmaier@xxxxxxxxxxxxx <mailto:stegmaier@xxxxxxxxxxxxx>>> wrote:
>>>>>> 
>>>>>> For now it would probably be fine to just restore the pragma for the for loop optimisation.
>>>>>> Mac users are used to work single-threaded, all others would get back multithreading here.
>>>>>> 
>>>>>>> On 1. Mar 2018, at 15:58, Tomasz Wlostowski <tomasz.wlostowski@xxxxxxx <mailto:tomasz.wlostowski@xxxxxxx><mailto:tomasz.wlostowski@xxxxxxx <mailto:tomasz.wlostowski@xxxxxxx>>> wrote:
>>>>>>> 
>>>>>>> On 01/03/18 15:43, Jeff Young wrote:
>>>>>>>> The purpose is it works on Mac.
>>>>>>>> 
>>>>>>>> But it does appear I misread the std::max( omp_get_num_procs(), 2 ) part.
>>>>>>>> 
>>>>>>> 
>>>>>>> Thanks Jeff!
>>>>>>> 
>>>>>>> Be aware that neither std::thread nor std::async have any concept of
>>>>>>> thread pooling - we need to look for a suitable library or write or own.
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> Tom
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> Mailing list: https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers> <https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>>
>>>>>>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx> <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>>
>>>>>>> Unsubscribe : https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers> <https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>>
>>>>>>> More help   : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp> <https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>>
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers> <https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>>
>>>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx> <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>>
>>>> Unsubscribe : https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers> <https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>>
>>>> More help   : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp> <https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>>
>>>> 
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers> <https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>>
>>>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx> <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>>
>>>> Unsubscribe : https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers> <https://launchpad.net/~kicad-developers <https://launchpad.net/~kicad-developers>>
>>>> More help   : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp> <https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>>
>>> 
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~kicad-developers
>>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~kicad-developers
>>> More help   : https://help.launchpad.net/ListHelp
>> 
>> 
>> 
>> 
>> _______________________________________________
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp
>> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp