Anthony’s Mac Labs Blog

📦 Multiple AutoPkg Recipes or Just Override?

Posted 2019 August 31

I’ve spent a lot of time digging in to the three primary sets of AutoPkg recipes that people commonly use to download Microsoft Office products. The more I dig, the more interesting learning and discussion points I find. Some of these are due to the craft of the authors. Some are due to differences in approach or need. And yet others are due to the chronology of events that led up to the recipes’ creation. I think you can learn a lot about writing AutoPkg recipes by just examining these recipe sets. It’s worthy of multiple blog posts, and will be an important part of my talk at the upcoming MacTech Conference in Los Angeles this October.

In this post, I want to focus on just one small detail: for a suite or family of applications, do you write one recipe that can download many products or write individual recipes for each product? What do the existing Microsoft product recipes do?

Variations on a Theme

Rich Trouton’s recipes take the one-recipe-per-product approach. In most cases, these recipes use a single template and then just change one default Input value (the FWLink number from Microsoft) to make it point to the desired product. There are some significant benefits to doing it this way.

First is ease of use. The recipe already contains the correct Input value for the product; the user doesn’t have to look it up like they would with a non-specific recipe. Second is discoverability. It allows you to put the specific product name in the recipe title and/or identifier (e.g., Excel, PowerPoint) so that people searching for a recipe (autopkg search) are more likely to find it. Then there are smaller benefits, like being able to add more recipes for more products without much effort (like Rich did when my previous post on this topic showed that there were three products he didn’t have recipes for yet). As well, some workflows really benefit from having named recipes — I’m thinking specifically of a shop like dataJAR, where they are supporting multiple clients, so they’re not just managing overrides for a single client.

A Kind of Master

The recipes in the core AutoPkg recipes repo take a clever hybrid approach: there is one master recipe (MSOfficeMacProduct.download.recipe), but individual download recipes also exist for all supported products. Take a look at one of those product download recipes and what do you notice? It is a child of the master download recipe which executes no additional steps; there is literally no Process section present!

This exposes something very interesting about how AutoPkg deals with variables declared in the Input section: if the same variable name appears in more than one recipe in the chain, it will adopt the value that appears last in the chain that you run. The core recipes leverage this to set the value of the PRODUCT variable to the one associated with the desired product.[1] So with this clever trick, there is only one master download recipe to maintain, but you get all the benefits mentioned above.

Of course, the child recipes further down the chain (.munki in this repo) have to be written using a template, but those recipes generally supply information for your management system that differs enough for each product to make it worth the effort. So this is still a pretty effective solution for the products that the core recipes support.

One Recipe to Rule Them All

But can we minimize maintenance even further? If I can write one recipe that can download different products or versions by simply changing a particular Input value in my override, then I only have to maintain that one recipe. This is an approach I have taken with a number of other recipes I have written. Sometimes, it’s about offering a different major version or a different level of functionality (e.g., standard vs. Pro) or both. In the case of Office, the fixed links for download of Office products are very similar, so you could definitely take this approach. In fact, for my own personal Office downloading, I have done just that.

I wrote a single download recipe (MicrosoftMacProductFWLink.download.recipe) that can download any Microsoft product with a FWLink value. I then added one child recipe in order to copy the downloaded installer package to a local directory (in the future, I may create a recipe to move the package into a management system and/or pick up version information). Since you can make multiple overrides of a single recipe[2], I made one override for the Suite and 5 different individual product recipe overrides. I changed the Input value for PRODUCTID to the FWLink value I need for that product in each override.

If Microsoft changes any of the FWLink values (as they did somewhat in the transition from Office 2016 to 2019), I can change the value of PRODUCTID in the override and I’m done. If Microsoft changes their URL scheme but continues basing it on FWLink values, then I can patch just one download recipe and be rolling again. Simple.

If there’s a down side to this method, it’s that it requires more operator skill. For the recipe author, that’s no problem (at least initially). If I want to share it, however, I have to document it clearly and trust that other users will read the documentation. As I mentioned earlier, it’s also hard to find if you are looking for, say, an Excel recipe using autopkg search, since that product name does not appear anywhere in the recipe (even though that’s the recipe I personally use for that product).

I want to be clear that I am not recommending that anyone adopt my method for Office recipes. There are many good reasons to use one of the existing sets of recipes, including the fact that I expect them to be maintained. I am just including them here as a proof of concept — the techniques may be useful to you when you are conceiving of how you might write recipes for other products.

Input Variables as Constants…

You may have notices that none of the individual Microsoft product recipes (found in the core recipes and Rich Trouton recipes repos) hardcode product-specific values into the Process section of the recipes. They use an Input variable as a kind of constant. In fact, the Rich Trouton and Allister Banks-hosted (arubdesu) recipes use the Microsoft download URL as such a constant as well. I actually wish we could declare constants in AutoPkg recipes. Using constants (that recipes users could not override) could make recipes more readable and, when used more than once in a recipe, easier to update. Coders do this instinctively, which is why we see this technique in many other AutoPkg recipes.

Nevertheless, because these are actually variables, users can override their value. This was a plus for the Office suite download recipe in the arubdesu-recipes repo when Microsoft moved from 2016 to 2019; one could simply change the value of a specific Input key to the updated FWLink number from Microsoft and get a (mostly) functional recipe. The down side of using variables as constants is that, when the recipe author updates an Input variable value in the original recipe, someone who has an override in place does not receive the updated value. This was a bigger deal before Trust Verification came into being, when many users would accept recipe updates from others willy nilly. (There aren’t any of those people left, are there?) Now users who are auditing changed recipes will still see that there is a change, but they will have to manually copy the changed value(s) over to their override or create a new override, which will then take the new values as default. [Update: A useful technique with constants is to remove them from your override, so that the author’s value is always used. More details can be found in this more recent post.]

On the flip side, moving those pseudo-constants out of the Input section and into the Process section as hardcoded values means that when you as an author update a recipe, you know that users (including yourself) will get the update next time they update their copy of your repo — no handstands required. It does mean that a pull request or an an update by the maintainer(s) is required for a fix, however. We’ve already seen in the past two transitions of Office recipes that certain recipes didn’t get updated until the maintainers had access to the new products and needed to deploy them, so both approaches have their liabilities. In general, we are seeing recipe authors move away from putting URLs and regular expressions in the Input section as variables.

Someone using the FWLink download method could reasonably argue that yet another set of Office recipes should be written with fixed links in the Process section for every product, as this is the most fault-resistant method (if you want full installers), but I don’t think there’s any reason to muddy the waters now. We don’t expect any significant changes from Microsoft until they release their next perpetual version of Office in, say 2022, so as long as users understand what these recipes are doing when the next change comes along, we’ll be fine.

More To Learn

For those who follow me on Twitter, you know that every time I thought I was done a post, I kept finding something worth further comment in these recipes. The post I was working on at the time wasn’t this one — I determined that I needed to start a new post from scratch for this topic. So I’ve got one or two more blog posts to come on these recipes, including the pros and cons of using custom processors versus just the core processors. I look forward to bringing you those soon.

Updated 2019-11-11 to add another technique for dealing with constants.

[1] In fact, you can’t even run the parent recipe on its own, as the default value for PRODUCT is not considered to be valid by the custom processor. [Return to main text]

[2] Make certain you change the identifier and filename of each override to make them unique. If you’re using AutoPkgr to make your override, change the recipe name when prompted as you create the override; when using autopkg make-override, use the -n flag to specify the recipe name. In both methods, the identifier and recipe name will both be set based on the value supplied. [Return to main text]