kicad-developers team mailing list archive

[Tomasz Wlostowski <tomasz.wlostowski@xxxxxxx>]

On 01/03/18 15:01, Jon Evans wrote:
> I missed that change to STL, oops!
> In that case we should standardize on STL for all new code in the future
> so that it will work on Mac.

Me too.

I don't understand the purpose of it - now all zone calculation (except
for insulated copper island removal) works in a *single* worker thread
(the main thread is just refreshing the progress dialog).

It means ~3.5x slower zone refilling on a 4-core CPU.

Can anybody comment on that?

Tom


> There are already many edge cases I have found (or users have reported)
> where KiCad is extremely slow, and we need to have a good story for
> multiprocessing so that we can take advantage of it where it makes sense
> to increase performance and/or responsiveness.
> 
> -Jon
> 
> On Thu, Mar 1, 2018 at 8:48 AM, Jeff Young <jeff@xxxxxxxxx
> <mailto:jeff@xxxxxxxxx>> wrote:
> 
>     There’s performance and there’s responsiveness.
> 
>     The footprint loading and zone filling were moved to STL so that
>     progress reporting would work on Mac.
> 
>     As long as we don’t introduce more OpenMP into core stuff, I’m not
>     that fixated on what we use for raytracing.
> 
>     Cheers,
>     Jeff.
> 
> 
>>     On 1 Mar 2018, at 13:34, Tomasz Wlostowski
>>     <tomasz.wlostowski@xxxxxxx <mailto:tomasz.wlostowski@xxxxxxx>> wrote:
>>
>>     On 01/03/18 14:29, Jon Evans wrote:
>>>     It is also used for zone filling and in my new eeschema connectivity
>>>     code that I'm hoping to get merged soon after V5 release.
>>>
>>>     I'm happy to help with research / testing of alternatives if we can't
>>>     use OpenMP on Mac. 
>>>
>>     I'd like to hear from a *single* Apple user who designs boards of
>>     sufficient complexity for the multithreading to really have an
>>     impact on
>>     performance. What's next, rewriting Kicad in Objective-C?
>>
>>     - my 5 cents
>>
>>     Tom
>>
>>
>>>     Jon
>>>
>>>     On Thu, Mar 1, 2018, 08:26 Wayne Stambaugh <stambaughw@xxxxxxxxx
>>>     <mailto:stambaughw@xxxxxxxxx>
>>>     <mailto:stambaughw@xxxxxxxxx>> wrote:
>>>
>>>        Ughh!  Thank you Apple.  I'm not opposed to this idea if
>>>     OpenMP isn't a
>>>        viable option on macos.  I would like to see our macos port be
>>>     on par
>>>        with linux and windows ports.
>>>
>>>        Wayne
>>>
>>>        On 3/1/2018 8:20 AM, Bernhard Stegmaier wrote:
>>>>     I did look around a bit.
>>>>     Looks like OpenMP isn’t a friend of Xcode clang, because Apple
>>>>     has its
>>>>     own Grand Central Dispatch (libdispatch) implementation/library
>>>>     doing
>>>>     similar things (at first glance, didn’t look into details).
>>>>     So, I guess OpenMP maybe won’t happen soon with stock toolchain
>>>>     or we
>>>>     would have to use non-stock toolchain (probably with all
>>>>     problems that
>>>>     come with that - I have seen bad things mixing compilers, etc.).
>>>>
>>>>     Seems to be quite easy to convert OpenMP for loop parallelisation
>>>        to GCD:
>>>>     <<<
>>>>     //#pragma omp parallel for
>>>>     //for( signed int i = 0; i < aPolySet.OutlineCount(); ++i )
>>>>     dispatch_queue_t queue =
>>>>     dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
>>>>     dispatch_apply(aPolySet.OutlineCount(), queue, ^(size_t i)
>>>>     {
>>>>     ...
>>>>     }
>>>>     );
>>>>>>>
>>>>
>>>>     I just naively replaced all of the OpenMP for loops like above
>>>>     and now
>>>>     3d-Viewer uses all cores.
>>>>     On my i7-3770T rendering time decreased from about 100s to 30s.
>>>>
>>>>     I also did it for the ratsnest loop but didn’t notice any difference
>>>>     (maybe my test PCB was just to small).
>>>>
>>>>     I could play around a bit to put that stuff into some nice syntax
>>>        maybe
>>>>     similar to what Microsoft has with concurrency::parallel_for_each
>>>>       https://msdn.microsoft.com/en-us/library/dd492857.aspx
>>>>     <https://msdn.microsoft.com/en-us/library/dd492857.aspx>
>>>>     So, at least ugly #ifdef around each parallel for loop could be
>>>        hidden.
>>>>
>>>>     Don’t get me wrong, I don’t want to add just another parallelisation
>>>>     library/mechanism.
>>>>     And… I guess it won’t be possible to transparently cover all the
>>>        OpenMP
>>>>     pragma stuff maybe needed inside for loop body or maybe other stuff
>>>>     needed for GCD (e.g. when writing to variables outside for body).
>>>>
>>>>     Thoughts? Would this be an option?
>>>>
>>>>     Oh, yes:
>>>>     I did look into using STL std::async.
>>>>     Doesn’t seem to be a good idea, because it usually doesn’t seem
>>>>     to use
>>>>     anything like thread pools, etc. on various platforms.
>>>>     You probably don’t want to create that much threads… :) 
>>>>
>>>>
>>>>     Regards,
>>>>     Bernhard
>>>>
>>>>>     On 1. Mar 2018, at 08:40, Bernhard Stegmaier
>>>        <stegmaier@xxxxxxxxxxxxx
>>>     <mailto:stegmaier@xxxxxxxxxxxxx> <mailto:stegmaier@xxxxxxxxxxxxx
>>>     <mailto:stegmaier@xxxxxxxxxxxxx>>
>>>>>     <mailto:stegmaier@xxxxxxxxxxxxx <mailto:stegmaier@xxxxxxxxxxxxx>
>>>        <mailto:stegmaier@xxxxxxxxxxxxx
>>>     <mailto:stegmaier@xxxxxxxxxxxxx>>>> wrote:
>>>>>
>>>>>     I had a quick look where OpenMP is used.
>>>>>
>>>>>     I found it somewhere in the connectivity/ratsnest code.
>>>>>     I don’t know of any complaints in that area and even on my old HW I
>>>>>     had never a bad experience.
>>>>>
>>>>>     And then, 3D-Viewer.
>>>>>
>>>>>     So, in my opinion it is basically only about 3D-Viewer… I don’t
>>>>>     know
>>>>>     if user experience will be that bad without OpenMP.
>>>>>     IMHO it’s only the Raytracing view, which isn’t intended to be used
>>>>>     for normal viewing stuff anyway.
>>>>>
>>>>>     If I find some time over the weekend I will also try… but I don’t
>>>>>     think it will be worth it.
>>>>>
>>>>>
>>>>>     Regards,
>>>>>     Bernhard
>>>>>
>>>>>>     On 1. Mar 2018, at 00:50, Jeff Young <jeff@xxxxxxxxx
>>>>>>     <mailto:jeff@xxxxxxxxx>
>>>        <mailto:jeff@xxxxxxxxx>
>>>>>>     <mailto:jeff@xxxxxxxxx ;<mailto:jeff@xxxxxxxxx
>>>>>>     <mailto:jeff@xxxxxxxxx>>>> wrote:
>>>>>>
>>>>>>     Or rip it out and use STL?
>>>>>>
>>>>>>>     On 28 Feb 2018, at 23:38, Jon Evans <jon@xxxxxxxxxxxxx
>>>>>>>     <mailto:jon@xxxxxxxxxxxxx>
>>>        <mailto:jon@xxxxxxxxxxxxx>
>>>>>>>     <mailto:jon@xxxxxxxxxxxxx ;<mailto:jon@xxxxxxxxxxxxx
>>>>>>>     <mailto:jon@xxxxxxxxxxxxx>>>> wrote:
>>>>>>>
>>>>>>>     Hi all,
>>>>>>>
>>>>>>>     Does anyone have a working build setup for getting OpenMP-enabled
>>>>>>>     KiCad out of MacOS?
>>>>>>>     If so, please share how -- I tried for a bit but couldn't get it
>>>>>>>     going (I'm not super familiar with the MacOS toolchain yet).
>>>>>>>
>>>>>>>     We should make sure that the 5.0 release is built with OpenMP,
>>>>>>>     otherwise our Mac users are going to have a slower user
>>>>>>>     experience.
>>>>>>>
>>>>>>>     -Jon
>>>>>>>     _______________________________________________
>>>>>>>     Mailing list: https://launchpad.net/~kicad-developers
>>>>>>>     <https://launchpad.net/~kicad-developers>
>>>>>>>     Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>>>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>>        <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>>
>>>>>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>>>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>>        <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>>>
>>>>>>>     Unsubscribe : https://launchpad.net/~kicad-developers
>>>>>>>     <https://launchpad.net/~kicad-developers>
>>>>>>>     More help   : https://help.launchpad.net/ListHelp
>>>>>>>     <https://help.launchpad.net/ListHelp>
>>>>>>
>>>>>>
>>>>>>     _______________________________________________
>>>>>>     Mailing list: https://launchpad.net/~kicad-developers
>>>>>>     <https://launchpad.net/~kicad-developers>
>>>>>>     Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>>        <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>>
>>>>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>>        <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>>>
>>>>>>     Unsubscribe : https://launchpad.net/~kicad-developers
>>>>>>     <https://launchpad.net/~kicad-developers>
>>>>>>     More help   : https://help.launchpad.net/ListHelp
>>>>>>     <https://help.launchpad.net/ListHelp>
>>>>>
>>>>>
>>>>>     _______________________________________________
>>>>>     Mailing list: https://launchpad.net/~kicad-developers
>>>>>     <https://launchpad.net/~kicad-developers>
>>>>>     Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>>        <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>>
>>>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>>        <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>>>
>>>>>     Unsubscribe : https://launchpad.net/~kicad-developers
>>>>>     <https://launchpad.net/~kicad-developers>
>>>>>     More help   : https://help.launchpad.net/ListHelp
>>>>>     <https://help.launchpad.net/ListHelp>
>>>>
>>>>
>>>>
>>>>     _______________________________________________
>>>>     Mailing list: https://launchpad.net/~kicad-developers
>>>>     <https://launchpad.net/~kicad-developers>
>>>>     Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>>        <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>>
>>>>     Unsubscribe : https://launchpad.net/~kicad-developers
>>>>     <https://launchpad.net/~kicad-developers>
>>>>     More help   : https://help.launchpad.net/ListHelp
>>>>     <https://help.launchpad.net/ListHelp>
>>>>
>>>
>>>        _______________________________________________
>>>        Mailing list: https://launchpad.net/~kicad-developers
>>>     <https://launchpad.net/~kicad-developers>
>>>        Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>>        <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>>
>>>        Unsubscribe : https://launchpad.net/~kicad-developers
>>>     <https://launchpad.net/~kicad-developers>
>>>        More help   : https://help.launchpad.net/ListHelp
>>>     <https://help.launchpad.net/ListHelp>
>>>
>>>
>>>
>>>     _______________________________________________
>>>     Mailing list: https://launchpad.net/~kicad-developers
>>>     <https://launchpad.net/~kicad-developers>
>>>     Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>>     Unsubscribe : https://launchpad.net/~kicad-developers
>>>     <https://launchpad.net/~kicad-developers>
>>>     More help   : https://help.launchpad.net/ListHelp
>>>     <https://help.launchpad.net/ListHelp>
>>>
>>
>>
>>     _______________________________________________
>>     Mailing list: https://launchpad.net/~kicad-developers
>>     <https://launchpad.net/~kicad-developers>
>>     Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>     Unsubscribe : https://launchpad.net/~kicad-developers
>>     <https://launchpad.net/~kicad-developers>
>>     More help   : https://help.launchpad.net/ListHelp
>>     <https://help.launchpad.net/ListHelp>
> 
>