kicad-developers team mailing list archive

[Jeff Young <jeff@xxxxxxxxx>]

Awesome!  Great job, Adam.


> On 4 Feb 2020, at 22:03, Adam Wolf <adamwolf@xxxxxxxxxxxxxxxxxxxx> wrote:
> 
> Hi folks!
> 
> Good news.
> 
> On my dev machine, I've got the DMG, the apps, all the frameworks,
> everything, all notarized.  The apps are all signed even!
> 
> We don't need to change the symlinks! (I'm a little bummed that the
> relayed question->answer from WWDC wasn't accurate, but that was 3
> years ago, and there have been changes to how notarization is enforced
> as recently as 2020/02/03. :) )
> 
> What was it?  A combination of two things:
> 
> * Moving some Python and ngspice things around in the bundle so that
> only certain types of executables were in certain directories of the
> bundle
> * Apple says to submit a zip to their notarization service.  If you
> create a zip with "zip", it won't work. You need to use ditto with a
> few different arguments.  I suspect that this was messing with the
> symlinks.  *sigh*
> 
> Alright, so what's next?
> 
> 1) I want to remove all the KiCad patches from kicad-mac-builder,
> long-term.  As part of that, there's a few changes from this effort
> that I want to get into the KiCad cmake files.  I think they're
> innocent and gentle but we can discuss them on the MRs I'll make. :)
> However, I have an old patch to create a distributable Python
> Framework that I don't really want to merge into our tree--and I am
> personally OK with leaving it as a patch in kicad-mac-builder, since
> a) it has been that way since I created it years ago and 2) my next
> effort after this notarization/signing is 100% done is getting Python
> 3 working on the macOS builds, and as part of that I'll be getting rid
> of this patch.  At that point kicad-mac-builder won't have any KiCad
> patches at all.
> 
> 2) It is certainly possible that I broke things or made minor mistakes
> with signing/notarization, so I will generate some of these builds on
> my dev machine and get them available as testing builds for people to
> test.
> 
> 3) It is possible there are ways that users use the apps now that will
> break.  For instance, it will no longer be OK for users to put their
> own files inside the bundle.  This is a good thing, but we may need to
> prepare some instructions or something to help break it to users--if
> anyone has strong opinions here please let me know.  I am terrified
> that users are doing this, downloading a new version of KiCad,
> overwriting their old "executables" and then losing their data files.
> I have read of at least one report of this happening on kicad.info and
> it breaks my heart.   I know there has been some confusion about this
> point technically but see
> https://developer.apple.com/library/archive/technotes/tn2206/_index.html#//apple_ref/doc/uid/DTS40007919-CH1-TNTAG304
> for details.
> 
> 4) I have some kicad-mac-builder changes to make to get all of this
> signing and notarization stuff working on CI.  I expect this will take
> at least a few days as the part I need to modify the most is a part
> that's been a pain lately and needs to be gutted anyway...
> 
> 5) When all of this is done, and this is stable, Python 3 on macOS is
> next on my plate.
> 
> Thanks folks!
> 
> If anyone has any questions or comments or opinions on this stuff, let
> me know, otherwise I'll keep at it :)
> 
> Adam Wolf
> 
> On Mon, Feb 3, 2020 at 10:05 AM Bernhard Stegmaier
> <stegmaier@xxxxxxxxxxxxx> wrote:
>> 
>> Adam,
>> 
>> One last thing, just to be sure… you are notarizing the final kicad.app after the “make install”?
>> If you use the (intermediate) one of the build directory, the sub-bundles might not yet be in place as they should…
>> 
>> 
>> Bernhard
>> 
>>> On 3. Feb 2020, at 16:59, Bernhard Stegmaier <stegmaier@xxxxxxxxxxxxx> wrote:
>>> 
>>> OK, I see… this is weird.
>>> My understanding from what I read in the documentation was, that it should sign everything in the top-level bundle (kicad.app) including sub-apps you might have in kicad.app/Contents/Applications.
>>> 
>>> 
>>> Regards,
>>> Bernhard
>>> 
>>>> On 3. Feb 2020, at 16:47, Adam Wolf <adamwolf@xxxxxxxxxxxxxxxxxxxx> wrote:
>>>> 
>>>> I am not notarizing the DMGs.  While this is possible, it has not been
>>>> necessary so far.
>>>> 
>>>> When I tried notarizing kicad.app but not the others, when I move to a
>>>> new computer, it complains that eeschema.app is not notarized.
>>>> 
>>>> The problem is not putting the signed kicad.app into an unsigned dmg.
>>>> 
>>>> On Mon, Feb 3, 2020 at 8:11 AM Bernhard Stegmaier
>>>> <stegmaier@xxxxxxxxxxxxx> wrote:
>>>>> 
>>>>> Hi Adam,
>>>>> 
>>>>> I am also no fan of the symlinks, but having a different approach will
>>>>> be probably some work.
>>>>> 
>>>>>> I had someone ask if what we do would work during WWDC and I was told
>>>>>> it would not work.  I consistently get "the signature is invalid" when
>>>>>> signing while we have symlinks, and when I remove the symlinks and
>>>>>> just sign KiCad.app this error goes away.
>>>>> 
>>>>> I don't doubt that the symlinks in the DMG don't work.
>>>>> What you explained is exactly what I had in mind:
>>>>> (1) Sign *only* kicad.app as is. No complete DMG with symlinks or
>>>>> whatever.
>>>>> (2) Create DMG with previously signed kicad.app and symlinks, libraries
>>>>> and whatever you put into. Don't try to notarize this DMG, DMG is just
>>>>> a container.
>>>>> 
>>>>> Doesn't that work?
>>>>> kicad.app is signed and the DMG should just acts as some kind of zip
>>>>> file then... ?
>>>>> 
>>>>> If the problem is putting the signed kicad.app into a (unsigned) DMG,
>>>>> maybe just distributing via .zip would be also a viable way meanwhile?
>>>>> Many other applications also do this...
>>>>> 
>>>>> 
>>>>> Regards,
>>>>> Bernhard
>>>>> 
>>>>> Am 3.2.2020 14:46, schrieb Adam Wolf:
>>>>>> Bernhard,
>>>>>> 
>>>>>> I have no personal vendetta against the symlinks.
>>>>>> 
>>>>>> I had someone ask if what we do would work during WWDC and I was told
>>>>>> it would not work.  I consistently get "the signature is invalid" when
>>>>>> signing while we have symlinks, and when I remove the symlinks and
>>>>>> just sign KiCad.app this error goes away.
>>>>>> 
>>>>>> I am not sure if Apple gives themselves special entitlements that mere
>>>>>> mortals don't get.  I'm not sure if I'm just not able to get it to
>>>>>> work.
>>>>>> 
>>>>>> Nothing I have done so far relies on the symlinks going away, so if
>>>>>> you think you can make it work, please let me know.
>>>>>> 
>>>>>> My personal suggestion for working around the symlinks issue was not
>>>>>> to just copy things, but rather just have a single KiCad.app that
>>>>>> would open itself in different ways of given a different type of file,
>>>>>> but others on the bug tracker preferred trying to copy things first.
>>>>>> 
>>>>>> Frankly, it's exhausting spending all this time on things that users
>>>>>> don't see, when there are so many interesting fun things we could be
>>>>>> working on instead.
>>>>>> 
>>>>>> In terms of what I am signing and notarizing, I have tried signing and
>>>>>> notarizing the app, the dmg, all the apps, basically every
>>>>>> combination.  Apple's rules are extremely fickle here, and you could
>>>>>> even notarize unsigned things.  They explicitly say the rules about
>>>>>> what you can notarize are hidden from developers!
>>>>>> 
>>>>>> Adam
>>>>>> 
>>>>>> On Mon, Feb 3, 2020, 1:08 AM Bernhard Stegmaier
>>>>>> <stegmaier@xxxxxxxxxxxxx> wrote:
>>>>>> 
>>>>>>> Hi Adam,
>>>>>>> 
>>>>>>> I still don’t get it:
>>>>>>>> Our current
>>>>>>>> strategy of symlinking into the kicad.app bundle does not work
>>>>>>> with
>>>>>>>> macOS signing.
>>>>>>> 
>>>>>>> Xcode has e.g. Instruments application in
>>>>>>> Xcode.app/Contents/Applications/Instruments.app
>>>>>>> If I symlink it (for example) to
>>>>>>> /Applications/Instruments.app
>>>>>>> It runs without any complaints when started via the symlink.
>>>>>>> 
>>>>>>> What do you notarize?
>>>>>>> The overall dmg with the symlink?
>>>>>>> Have you already tried to only notarize kicad.app (no dmg, no
>>>>>>> symlinks) and put it into the dmg with symlinks afterwards?
>>>>>>> Another quick fix could be some script that can be run to create the
>>>>>>> symlinks on user machine?
>>>>>>> 
>>>>>>> A simple copy of the apps won’t work.
>>>>>>> You need to change everything wrt shared libraries in KiCad code and
>>>>>>> cmake script.
>>>>>>> 
>>>>>>> In the end, you will duplicate all libraries and support stuff.
>>>>>>> Probably not a big deal for eeschema and the other small apps, but I
>>>>>>> guess for pcbnew.
>>>>>>> Means duplicating all the python, nags-ice, etc. stuff.
>>>>>>> And also, all stuff like templates, scripts, etc.
>>>>>>> Users shouldn’t fiddle around in the .app, but could get really
>>>>>>> messy if they now put (template, python, spice?) stuff in kicad.app
>>>>>>> or pcbnew.app and then something doesn’t work in one or the
>>>>>>> other...
>>>>>>> 
>>>>>>> Regards,
>>>>>>> Bernhard
>>>>>>> 
>>>>>>>> On 3. Feb 2020, at 02:00, Adam Wolf
>>>>>>> <adamwolf@xxxxxxxxxxxxxxxxxxxx> wrote:
>>>>>>>> 
>>>>>>>> Hi folks!
>>>>>>>> 
>>>>>>>> Apple is changing how the lack of notarization looks like to users
>>>>>>> on
>>>>>>>> Catalina starting tomorrow.  It is not clear what will happen when
>>>>>>>> folks download new versions of KiCad after tonight.
>>>>>>>> 
>>>>>>>> For the past two months I've been working hard--I've got a tech
>>>>>>> demo
>>>>>>>> locally here that has signatures and notarization on macOS, but
>>>>>>> it's
>>>>>>>> not ready for primetime.  For instance, I have removed the other
>>>>>>> .apps
>>>>>>>> and just have kicad.app.  There's changes I made to kicad that
>>>>>>>> probably belong in kicad-mac-builder--and, well, let's just say
>>>>>>> it's a
>>>>>>>> tech demo :)
>>>>>>>> 
>>>>>>>> The main things that remain are:
>>>>>>>> 1) Figure out a good solution for the symlinked .apps.  Our
>>>>>>> current
>>>>>>>> strategy of symlinking into the kicad.app bundle does not work
>>>>>>> with
>>>>>>>> macOS signing.  I think the current contender is to copy instead
>>>>>>> of
>>>>>>>> symlink.  I am not sure how much extra space that will take up but
>>>>>>>> it's a good try.  This is definitely something I can do, but since
>>>>>>>> it's something that can be done on its own, it's a prime contender
>>>>>>> for
>>>>>>>> someone looking to help out.
>>>>>>>> 
>>>>>>>> 2) Another issue is that there are strict rules about where in the
>>>>>>>> bundle code, data, and executable non-Mach-O files live.  For
>>>>>>>> instance, one of the signing blockers is ngspice, because it
>>>>>>> mingles
>>>>>>>> scripts and Mach-O binaries and then we put them in
>>>>>>> Contents/Plugins.
>>>>>>>> For more details, see
>>>>>>>> 
>>>>>>> 
>>>>>> https://developer.apple.com/library/archive/technotes/tn2206/_index.html#//apple_ref/doc/uid/DTS40007919-CH1-TNTAG201.
>>>>>>>> The big change for KiCad itself is where the Python scripts are
>>>>>>>> stored--I've fixed this in my branch, but now I have to go through
>>>>>>> and
>>>>>>>> audit and fixup our partner packages, like OCE/OCC and ngspice.
>>>>>>> If
>>>>>>>> you want to help with this, it's going to be a big job but I'm
>>>>>>> willing
>>>>>>>> to put in the time to teach if you're willing to put in the time
>>>>>>> to
>>>>>>>> learn :)
>>>>>>>> 
>>>>>>>> I was really hoping I could get this done before Apple turned up
>>>>>>> the
>>>>>>>> enforcement on notarization, but that's going to happen.  After
>>>>>>>> tomorrow, it'll be clearer what Apple is doing.  There might be
>>>>>>> some
>>>>>>>> quick changes to make that will improve things for our users
>>>>>>> without
>>>>>>>> getting all of this done.
>>>>>>>> 
>>>>>>>> Adam Wolf
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> Mailing list: https://launchpad.net/~kicad-developers
>>>>>>>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>>>>>> Unsubscribe : https://launchpad.net/~kicad-developers
>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>> 
>> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp