Posted 2021 April 26
This is one of the simpler things I’ve learned about how AutoPkg works, but I’m sharing it here in case it happens to meet a use case of yours. When setting arguments for a processor, you can set the value of a variable to an empty string (<string/> or <string></string>) to ensure that the processor ignores any value for that variable that had been previously set in the recipe (e.g., as output from another processor). Let me explain how that might be useful.
The Principle of Persistence
In AutoPkg, once a variable exists within the run of a recipe, it persists until the end of the run. (It inherits this property from the Python language in which it is built.) This has potentially beneficial side effects. For example, a common start to a download recipe is to call a processor that fetches the URL of the product to be downloaded (e.g., custom processor, URLTextSearcher) immediately followed by the URLDownloader processor. In some cases, that URLDownloader processor is called without any arguments. But how can that be? The processor-info for URLDownloader states that the url variable is required.
In those instances, it’s because the previous processor returned the URL in an output variable named url. The URLDownloader processor doesn’t ask, “has the user added the url argument to the Arguments dictionary?” Instead, it asks, “what is the value of the variable url?” If that variable has been assigned a value any time prior to the processor call (including as an argument to that processor), AutoPkg will use it. (This applies whether the input variable is optional or required.) This can make recipes less verbose, since processor authors often create output variable names that pair nicely with input variable names of other processors.
When Persistence is Problematic
Having said this, this persistence could also lead to an unanticipated result in a later processor if you didn’t realize that a particular variable had been set. Here’s a concrete example using the recipes available for MeshLab. The download recipe (from the hansen-m-recipes repo) uses the GitHubReleasesInfoProvider processor to obtain the download URL. That processor returns three output variables: release_notes, url, and version. So when the next processor, URLDownloader, looks for the url variable, it finds that the value it needs has already been set.
I need a pkg installer to deploy MeshLab with my management system, so I created a pkg child recipe. Because the app is nicely contained in a disk image, I used the AppPkgCreator processor to do all the work. AppPkgCreator will not only bundle the app in a pkg installer but it will also grab the version number from the app to use in naming and versioning the package. But here’s where the side effect comes in. If you specify the value for version (as an Argument to the processor, for instance), the processor will use that instead. In this case, since the GitHubReleasesInfoProvider processor set the value for version, that is what AppPkgCreator will use for the version number; it will not try to extract it from the app itself.
Often, this does not cause a problem because the value would be the same. In the case of MeshLab, however, GitHubReleasesInfoProvider considers the version to be “MeshLab-” followed by what we think of as the version number. So when AppPkgCreator gets called, it uses that as the version number. That’s not what I want. Initially, my solution was to use Elliot Jordan’s FindAndReplace shared processor to get rid of the “MeshLab-” in the version number just before calling AppPkgCreator. A kind pull request from James Reynolds suggested using the AppDmgVersioner processor to grab the correct version instead. Both of these work, but I knew that AppPkgCreator had a versioning routine built in. So I wondered what would happen if I didn’t implement either of those solutions but instead set the value of version to an empty string in the Arguments section of the AppPkgCreator processor call, like so:
<dict> <key>Processor</key> <string>AppPkgCreator</string> <key>Arguments</key> <dict> <key>version</key> <string/> </dict> </dict>Or, if you’re using YAML recipes:
- Processor: AppPkgCreator
Arguments:
version: ''
As it turned out, AppPkgCreator interpreted a null value for version in the same way as if version had never been set. As I am beginning to learn, this aligns with the truthiness principle of the Python language, where strings are evaluated as True if they contain any characters and False if they are completely empty. In this case, clearing the value for version made the processor run its internal routine to grab the version number from the app bundle. This became the cleanest way to make certain I obtained the correct version number for my pkg.
Variables are (still) Arbitrary
This principle is a good corollary to the one I spoke about at the 2020 Mac Admins Campfire Sessions: variables are arbitrary. The Arguments dictionary for each processor is simply a way to establish variables and their value. For readability, we pair those Arguments with the processor that is going to use them. But variables are also created/set from the output of processors. AutoPkg will use anything that is available in the environment in which it is running. What I learned this month was that I could eliminate a previous value using a null string to trigger (or avoid triggering) an option in a processor call. I hope that this discussion helps you both read recipes more easily (uncovering the “hidden” variables that processors use) and provides an additional technique for you to use, even if just for debugging.