macOS: Notarize in Script?

Question

Because the codesigning and archiving by Xcode is time-consuming, boring and problematic, I've always codesigned, archived and shipped my Developer ID signed macOS app using the command-line tools xcodebuild, codesign, etc. via my own script. Notarizing looks like it is going to be a major pain. Is it possible to add notarizing to my script?

Answer 1

Yes. Unfortunately, the official answer leaves some loose ends, for example this important tidbit from Quinn "the Eskimo". Here is how to do it:

One-Time Setups

Get an App-Specific Password

Decide on a name for your "app" of notarizing apps. I use the name of my product-shipping script, SSYShipProduct.pl because this is the "app" which will use this password. We shall refer to whatever name you compose as your-notarizing-name.

Browse to https://appleid.apple.com/account/manage, scroll to Security > App-Specific Password, and generate an App-Specific password for an app named your-notarizing-name. Copy the password that it gives you. We shall call that app-specific-password.

Add the App-Specific password to your macOS Keychain

Run this command to add the password you just created to your keychain:

security add-generic-password -a "your-apple-ID-email" -w "app-specific-password" -s "your-notarizing-name"

The -s parameter is the name that this item will have in your Keychain. I think you could actually use a different name, but in my mind it makes sense to use your-notarizing-name here too.

You can verify that it worked by searching in the Keychain Access application. However, be aware that new items are not listed in Keychain Access until after you quit and relaunch it.

Maybe, get the relevant itc-provider

If your Apple ID is associated with more than one Apple Developer Connection team (such as if you do contract work), you will need the itc_provider of the team for which this app should be notarized.

To find the itc_provider of your team, execute this command:

/Applications/Xcode.app/Contents/Applications/Application\ Loader.app/Contents/itms/bin/iTMSTransporter -m provider -u "your-apple-ID-email" -p "app-specific-password"

Scroll to the end of the output printed by this command and look at the Provider listing table. Copy the Short Name of the desired team. We shall call this "developer-team-itc-provider".

For each shipment (Scriptable!)

If you sign components of your app using the /usr/bin/codesign command line tool, each invocation of codesign must have the following new argument parameter , which tells codesign to sign with the so-called hardened runtime:

 `--options runtime` 

Conversely if your app is signed in Xcode, you must set the Build Setting Hardened runtime, available in Xcode 10 or later, to Yes in all executable component targets.

Other than that, your script should create a build of your app in Release configuration and codesign it, same as in pre-notarization days.

Upload to Apple Notary service

Your script should then archive your app to a .zip or .dmg. Note that this is an interim file which will only be uploaded to the Apple Notary service, not shipped.

Then, your script should compose a primary bundle ID value, which will be your app's bundle identifier with .zip or .dmg appended. Example: your-pbid-value = com.mycompany.YourApp.zip.

In what follows, your script will use altool, which is Apple's name for Application Loader Tool.

Your script should then run this command to get your .zip or .dmg notarized:

/usr/bin/xcrun altool --notarize-app --primary-bundle-id "your-pbid-value" --username "your-apple-id-email" --password "@keychain:your-notarizing-name" -itc_provider "developer-team-itc-provider" --file /path/to/YourApp.zip/or/YourApp.dmg --output-format "xml"

(Note that, in the above command, oddly, all argument names are preceded by two dashes except -itc_provider is preceded by only one dash. Also, if the scripting language you are using interpolates @ characters in strings, code it to prevent interpolation of @keychain).

After a minute or so, xcrun will exit and print to stdout some XML which, if your submission was accepted (note: not approved yet), will look like this example:

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict>     <key>notarization-upload</key>     <dict>         <key>RequestUUID</key>         <string>2ab59b26-19ec-4a30-84cf-6d2cb8d3c97e</string>     </dict>     <key>os-version</key>     <string>10.15.0</string>     <key>success-message</key>     <string>No errors uploading 'path/to/YourApp.zip'.</string>     <key>tool-path</key>     <string>/Applications/Xcode.app/Contents/Applications/Application Loader.app/Contents/Frameworks/ITunesSoftwareService.framework</string>     <key>tool-version</key>     <string>1.1.1138</string> </dict> </plist> 

All you really need out of there is that RequestUUID value. However, since I ship four apps often, and because it ruins my day when my shipping script fails without providing helpful error information, and because (see below) you are going to make another call which also returns interesting XML, I invested some time in adding to my script a subroutine which takes two parameters, XML and a key path, and returns the value of the XML at a given key path. In the case above, I call this subroutine to get the RequestUUID, and then again to get the success-message.

(My script is in Perl. Although there is available in CPAN a module named XML::Simple which can do this parsing in a line or two, it is marked by the maintainer as not for use in new designs. So, to avoid needing to install and wrangle with a real XML parser, I opted instead to use PlistBuddy as suggested in the comment by @khuttun. This was slightly painful also because, unfortunately, altool does not have an option to write its output to a file, and PlistBuddy is not documented to accept stdin. So my subroutine writes the stdout from altool to a temporary file, and then passes that temporary file's path to PlistBuddy. Kind of disgusting, but it works.)

Delete the Un-stapled zip package

At this point, I recommend that your script delete the .zip or .dmg file which it uploaded. Reason: That file was archived from a product which does not yet have your notarization ticket stapled to it. At the end of your script, you will create a new .zip or .dmg from a modified app which has the ticket. Deleting the file immediately prevents you from shipping an un-stapled app by mistake.

Wait in a Loop for Apple's Response

Your script can then start pestering Apple's server for your final results, by running this command in a loop along with with some sleep:

`/usr/bin/xcrun altool --notarization-info --username "your-apple-id-email" --password "@keychain:your-notarizing-name" --output-format "xml"

If your script runs this command immediately, it will get returned in stdout some xml which will look something like this example:

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict>     <key>notarization-info</key>     <dict>         <key>Date</key>         <date>2019-08-07T01:17:37Z</date>         <key>RequestUUID</key>         <string>4ba71353-9d99-4b52-b579-37f384717130</string>         <key>Status</key>         <string>in progress</string>     </dict>     <key>os-version</key>     <string>10.15.0</string>     <key>success-message</key>     <string>No errors getting notarization info.</string>     <key>tool-path</key>     <string>/Applications/Xcode.app/Contents/Applications/Application Loader.app/Contents/Frameworks/ITunesSoftwareService.framework</string>     <key>tool-version</key>     <string>1.1.1138</string> </dict> </plist> 

The significant key path in there is notarization-info:Status, whose value in progress means that Apple is still working on your submission. After a few minutes usually (Apple says "should be less than an hour", but I experienced times of up to three and a half hours on the USA holiday afternoon of 2019-Jul-04), altool will retturn to your script a different xml in stdout, something like this:

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict>     <key>notarization-info</key>     <dict>         <key>Date</key>         <date>2019-08-06T23:28:25Z</date>         <key>LogFileURL</key>         <string>https://osxapps-ssl.itunes.apple.com/itunes-assets/Enigma113/v4/f6/09/be/f609bee3-b031-323a-0987-d1f620a78758/developer_log.json?accessKey=1565410613_1722173034418364591_TvycjBAzd6FRTYGKZEFU6EwDfsws8Wa1MV%2FYnTiJ1zyOZamc%2FoeO5RMeIzZN669ZQJgO2Q4W48ipKNFO%2BQGuq%2FITXN8MQAetbNe90w9ogzqXbrzTHg%2FgYK89yvEFmiiRxhaVlZqLI93NBpY0hwBqXv2bvvlg%2FRCc%2BVaCNRJ%2BrnE%3D</string>         <key>RequestUUID</key>         <string>07fc3745-b0ff-4d1a-9b15-37f384717130</string>         <key>Status</key>         <string>success</string>         <key>Status Code</key>         <integer>0</integer>         <key>Status Message</key>         <string>Package Approved</string>     </dict>     <key>os-version</key>     <string>10.15.0</string>     <key>success-message</key>     <string>No errors getting notarization info.</string>     <key>tool-path</key>     <string>/Applications/Xcode.app/Contents/Applications/Application Loader.app/Contents/Frameworks/ITunesSoftwareService.framework</string>     <key>tool-version</key>     <string>1.1.1138</string> </dict> </plist> 

After some reverse-engineering, you see that, in each loop iteration, your script should parse the XML and break out of the loop whenever the value of Status is something other than in progress, or if you prefer, when LogFileURL is defined. Or if you prefer email triggers, your script can look for an email from Apple with subject line You can now distribute your Mac software..

UPDATE 2019-11-02

After having trouble with this step in my last couple shipments, and again today, I have now confirmed a bug in Apple's Notary Service. The bug is that the altool --notarization-info command will fail for 1-5 hours, returning nonzero exit codes, and in stdout an error code 1519 "Could not find the RequestUUID", as in the following example stdout:

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict>     <key>os-version</key>     <string>10.15.1</string>     <key>product-errors</key>     <array>         <dict>             <key>code</key>             <integer>1519</integer>             <key>message</key>             <string>Could not find the RequestUUID.</string>             <key>userInfo</key>             <dict>                 <key>NSLocalizedDescription</key>                 <string>Could not find the RequestUUID.</string>                 <key>NSLocalizedFailureReason</key>                 <string>Apple Services operation failed.</string>                 <key>NSLocalizedRecoverySuggestion</key>                 <string>Could not find the RequestUUID.</string>             </dict>         </dict>     </array>     <key>tool-path</key>     <string>/Applications/Xcode.app/Contents/SharedFrameworks/ContentDeliveryServices.framework/Versions/A/Frameworks/AppStoreService.framework</string>     <key>tool-version</key>     <string>4.00.1181</string> </dict> </plist> 

This is a bug because, of course my script did submit the Request UUID which it just received from Apple Notary Service, Apple should be able to find it and, furthermore, when I kept sending the command manually, after about 2 hours, suddenly, the command returned Success and continued to return Success with subsequent commands, and I got the Success email from Apple. This delay happened today with 7 different good Request UUIDs, the longest was 5 hours. Possibly, at this time, there is a 1-5 hour delay between Apple Notary Service creating and sending you a Request UUID, and it appearing in the database which Apple Notary Service uses to respond to notarization-info requests, so you get this false error. Very sad.

Since I have no control over when Apple assigns people to fix bugs, I have modified this stage of my script to parse the response from Apple and die only if the command returns nonzero exit status and the code of the first (index=0) product-errors array entry is not 1519. If you are using PlistBuddy to parse XML as I am, the key path for that is code should be product-errors:0:code. The loop in my script prints each time Error 1519 is received, so I can see what is going on, and of course, I've modified its while condition to not exit if the error code is 1519.

After so fixing my script, I had several apps to ship. Apple Notary Service treated the first one nicely: No Errors 1519, and Success after about two minutes. The next one, however, needed this new feature of my script. At time 09:54 (HH:mm) my script received the Request UUID from Apple. 20 seconds later, it sent the first altool --notarization-info query. The response was a false Error 1519. Subsequent queries also returned false Errors 1519, for almost 3 hours, through 12:44. Then, at 12:45, all of a sudden it received an in progress response. After 5 more in progress responses, at 12:47, finally, Success.

One more thing before leaving this topic: An hour after that request succeeded with no Errors 1519, a prior request from an hour ago suddenly started returning in progress and then a few minutes later, Success. Conclusion: Request UUIDs which get detoured into the Error 1519 morass are not queued FIFO with later Request UUIDs which might, by chance, avoid the Error 1519 detour. So, a better workaround might be to abandon a Request UUID after receiving one more Error 1519 responses and start over by re-uploading the app to Apple Notary Service and getting another Request UUID which you hope will work better. Of course, you'll get many emails during the next few hours as all of the Request UUIDs which you abandoned eventually succeed.

At any rate, now, on to the next step in the script…

Check Apple's Log file

Your script should parse out the value of the LogFileURL so it can check the log, because because even if notarization succeeds the Log file created by Apple might contain warnings. To get the Log file your script should, of course,

curl <LogFileURL-Value> 

The Log file is apparently JSON. Warnings or Errors are presented as an array, which is the value of key issues . So your script should parse that curl output with a JSON parser and if the value of key issues is a JSON null or an empty array, continue shipping.

Staple the Ticket to your App

This step is pretty easy…

xcrun stapler staple /path/to/YourApp.app 

Running this command will add to your app's package a new file: YourApp.app/Contents/CodeResources. This is apparently your notarization ticket. Note that this file is in addition to the file YourApp.app/Contents/_CodeSignature/CodeResources which is still there, and contains the code signature, the same as in pre-notarization days.

Verify the Ticket Stapling

But there is a better way to verify that your app now has a good ticket. Your script should now run (or re-run) a Gatekeeper check:

spctl -a -v /path/to/YourApp.app 

The result, in stderr, should be,

/path/to/YourApp.app: accepted source=Notarized Developer ID 

which is the same result as pre-notarization, except for the insertion of Notarized. Astute scripts will parse that stderr and abort shipping if the above words are not detected.

Zip and Ship

Now that the ticket has been added, your script can zip or dmg your .app again, but this time, ship it.

Answer 2

Here's a reusable and freely-licensed notarize & staple script for automated builds:

https://github.com/rednoah/notarize-app/blob/master/notarize-app

It'll run and wait and only exit once everything is done:

1. Run altool --notarize-app
2. Run altool --notarization-info periodically until notarization is complete
3. Run stapler staple