Anthony’s Mac Labs Blog

Posted 2024 April 01

Recently, I added a new tool (Nicholas Riley’s brightness) to my colour calibration workflow and thus wanted to add it to the package I had created manually that contained a couple of other colour-related tools and the Outset trigger I run on every login (as described in a previous blog post). I decided that this was getting involved enough that I should consider building it in a more automated fashion to make future changes easier. I am very comfortable with using the packaging functionality of AutoPkg and knew I could do it with that, but I thought this might be a good opportunity to give munkipkg a try, since this seemed to suit its use case. With that in mind, I tried using both munkipkg and AutoPkg for this task and compared both with the GUI-based process I had used previously, discovering the advantages and disadvantages of these methods.

pkgbuild

But before I describe what I learned there, I should mention pkgbuild, since it is built into the OS and is an underlying tool that all of these other tools leverage. Armin Briegel’s eBook Packaging for Apple Administrators goes into great detail about using pkgbuild; between that and the man page, you should be able to suss out enough information to use it directly to complete the kind of task I am discussing here. Having said this, this task is intricate enough that I want to abstract some of the work, and all the tools I am about to discuss do that well.

munkipkg

munkipkg is run from the command line, but you do a lot of the work in the Finder: you replicate the target directory structure in a location of your choosing and tell munkipkg to build the package to look like that, adding preinstall and/or postinstall scripts if desired. It does require Python 3, but you will already have a suitable version installed if you have installed Munki, AutoPkg, or Mac Admins Python.^[1]

munkipkg will happily create the default structure with the --create option. This consists of folders for payload, scripts, and build, as well as a build-info.plist template. That last file holds build information about the package, such as version number, identifier, and package name. (If you prefer, you can use a JSON or YAML file in its place with the same information.) I did use the --create option to create the initial framework, but since I also had an existing package upon which I was building, I leveraged the --import option to give myself a model from which I could learn (and freely copy).

I was only placing 4 files and one app bundle with this package, but since the files were deeply nested, I had to create a number of folders within the payload folder it created to complete the structure:

Once I had all of those files and folder in place inside my payload, I then modified the build-info.plist provided using a plain text editor to specify details about the package. The munkipkg ReadMe describes all the keys in that plist, but one important key that is relevant here is that the ownership key has a default value of recommended, which in this case means we do not have to manually change the ownership of the above files to root/wheel in our payload hierarchy; munkipkg will do that for us. Another convenient default: the package name includes the version number you specify in that plist (which makes sense, since version info is critical in Munki).

I also leveraged the ability to sign the package described in the munkipkg ReadMe. In my case, the signing authority was stored in the keychain of the computer, so I just had to add the following keys to the build-info.plist:

    <key>signing_info</key>
    <dict>
        <key>identity</key>
        <string>My Signing Authority Name</string>
    </dict>

I did not have a need for a preinstall or postinstall script in this instance, so I left the scripts folder empty.

With everything now in place, I went to Terminal, changed its current working directory to where I had munkipkg stored, and sent the command munkipkg path/to/folder_I_created_with_create_option, which generated the package and placed it in the build folder within that structure. I took a look at the package using Suspicious Package to see what else I had to tweak. While munkipkg did a great job of establishing the correct ownership, the permissions were simply copied from the file. So I had to ensure that all of my executables had execute permission set before building the package (e.g., using the chmod +x command in the Terminal for the items in path/to/payload/usr/local/bin and path/to/payload/usr/local/outset/loginevery).

All in all, it wasn’t too difficult to build what I needed with munkipkg. Importing an existing package that I understood was really helpful in figuring out how munkipkg works. A little tweak on the executable permissions of my files was all the tweaking required to get a functional package, and is the only main thing I need to remember when creating an updated version of this package.

AutoPkg

AutoPkg is rightly lauded for its ability to automate the download of the latest version of literally thousands of software titles. But one feature that is often underrated is its ability to build packages programmatically. In this case, only one of the items in the package I want to build would be something I might download in the traditional way AutoPkg would. While this is still not required in order to access the packaging functionality, I decided that I would leverage that download ability in order to access the “self-contained” versions of Tim Sutton’s customdisplayprofiles in either Intel or Apple Silicon versions. (You can see this download recipe in my jazzace-recipes repo. An Input variable lets you select which architecture you want via override.)

Building the directory structure above is done with the PkgRootCreator processor. Since you need to create the intermediary directories as well, the Processor gets a lot of Arguments:

<dict>
        <key>Processor</key>
        <string>PkgRootCreator</string>
        <key>Arguments</key>
        <dict>
                <key>pkgdirs</key>
                <dict>
                        <key>Applications</key>
                        <string>0775</string>
                        <key>Applications/Utilities</key>
                        <string>0755</string>
                        <key>usr</key>
                        <string>0755</string>
                        <key>usr/local</key>
                        <string>0755</string>
                        <key>usr/local/bin</key>
                        <string>0755</string>
                        <key>usr/local/outset</key>
                        <string>0755</string>
                        <key>usr/local/outset/login-every</key>
                        <string>0755</string>
                </dict>
                <key>pkgroot</key>
                <string>%RECIPE_CACHE_DIR%/payload</string>
        </dict>
</dict>

Next, the recipe needs to copy the five items into their place in the newly-created package payload structure. With customdisplayprofiles, I know the location of the downloaded item from the parent recipe, so I can move it in to place directly.^[2] With all the others, I need an Input variable whose value is the path to the file I want copied. For example,

<key>Input</key>
<dict>
    <key>BRIGHTNESS_PATH</key>
    <string>/Volumes/AutoPkgLibrary/AutoPkg/Files/brightness</string><
[...]

I then use variable substitution (e.g., %BRIGHTNESS_PATH%) to tell the Copier processor where to find the item. While a bit tedious, this allows me to store the items being packaged anywhere on my system. I don’t have to compile them in a particular location — AutoPkg does that for me. This could be quite advantageous if you are managing any of these items using Git or some other version control system.

The final step in the recipe is to use the PkgCreator processor to build the package. The pkg_request dictionary is where you supply the kind of information that munkipkg stores in the build-info.plist, such as the package name, identifier, and version. While you can hardcode these, I let the recipe user supply that information mostly via Input variable.^[3]

The major difference between munkipkg and AutoPkg in this step is that AutoPkg’s packaging code uses pkgbuild’s “preserve” ownership option for the files included rather than the “recommended” option that is the default in munkipkg. This means we need to add a number of chown dictionaries to the pkg_request argument. We don’t have to do every single directory, since the chown request will be treated as recursive (subdirectories will inherit the ownership and permissions). I’ve chosen to explicitly mention each executable in my recipe to ensure the correct result (and make the recipe easier to understand) but I could potentially eliminate some of those with more testing.

The complete recipe is available in my jazzace-recipes repo (JazzAce/DisplayCalibrationSupportFiles.pkg.recipe). If I wanted this package to be signed, I could add a .sign child recipe similar to one in my repo.

GUI-based tools

There are two tools of note if you want to use a GUI-based tool to build bespoke packages:

• WhiteBox Packages by Stéphane Sudre (free), and
• Composer by Jamf (included with Jamf Pro or US$100/$80 EDU when purchased separately).

Neither tool has received any significant updates in the past number of years, but they do still work.^[4] They both allow you to build a package from scratch by dragging the files into a List view window. In Packages, you create a project and then add files to the list. A number of common folders are provided to make it more of a drag-and-drop experience. In Composer, since its focus is snapshotting, you need to start a package from scratch by (counterintuitively) cancelling the “Choose a method to create your package” dialogue to arrive at the main window, then dragging one of your files into an open spot in the left sidebar. That will start a new package, which by default uses the filename of the item you dragged in (if you only drag one in) as the package name, and will list that file’s install location as being the same as where it was in the Finder (you can change all of this after the fact). You then drag further files in as needed. Both apps let you specify details about the package itself, like identifier and version, and you can alter permissions and ownership on each item. Packages has a couple of thoughtful touches (like asking whether you want to retain ownership when dragging-and-dropping a file into the list), but both are serviceable. Changing ownership and permissions is something that can be done fairly simply from the GUI, without having to resort to the Terminal (like munkipkg requires to ensure that scripts are executable, for instance).

The main difference between the two for this kind of recurring package job is that Packages only stores the path of the source file(s) in its project file, whereas Composer has a private cache and copies the files to that cache when added to the package. If you move or delete a source item in a Packages package, it won’t be able to build the package. This also mirrors the difference between my AutoPkg recipe (which stores paths to files in Input variables) and munkipkg (which caches all files in the payload).

One Task, Many Solutions

In the end, once I understood how munkipkg worked, it was the simplest to get right. The ability to use a pre-existing package as a model was very helpful. AutoPkg’s only advantage in this scenario is that it can more easily assemble the separate Intel and Apple Silicon package versions I need, as well as anything else under source control. Both are easy to update in the future. If I just needed a one-off, both Packages and Composer give me the level of control I need, and I could still come back to them in the future if I needed to make a small update.

But your use case may not be mine. For easy reference, I’ll conclude with a tabular summary of the key features of each tool. Hopefully, you can find the right packaging tool(s) for your own needs.

[1] In order for munkipkg to be able to find the installed version of Python 3 you wish it to use, you can change the shebang in the munkipkg code to point to it, change your environment variable to point python3 to that version, or specify which Python to run at the command line each time (e.g., /usr/local/bin/managed_python3 munkipkg […]). [Return to main text]

[2] I also rename the executable, since I want it to be named the same regardless of whether it is Intel or Apple Silicon. [Return to main text]

[3] Normally, we would let AutoPkg fetch the version number from the primary executable it downloaded. In this case, I want to version the compiled package (since the components have their own version numbers), so I make it an Input variable. This way, I can change it in my override or even at runtime (e.g., autopkg run DisplayCalibrationSupportFiles.pkg -k PKG_VERSION="2.0.3"). If you don’t even want that much hassle, I have a VersionGenerator shared processor that provides a (large) version number based on the current Universal time (in seconds), thus ensuring the version number is higher each time you run it. [Return to main text]

[4] I had to open Packages from a contextual menu when freshly installed on a Mac running Sonoma to override the error message saying it was damaged. All subsequent launches were without error. [Return to main text]