jazzace.ca

Anthony’s Mac Labs Blog

📦 AutoPkgr Repo List Limits

Posted 2018 September 08

I’ve been working on a short series of posts on AutoPkg that would share some of the things I didn’t have the time to discuss during my session at the 2018 MacAdmins Conference at Penn State; I’ve already posted one. But something I discovered this week is worth documenting, so it’s jumping the queue. It only affects AutoPkgr users, and you can work around it, but it is unexpected.

TL;DR

The list of repos in the AutoPkgr Repos & Recipes tab does not show all the recipe repos available within the AutoPkg project. You can still add a “missing” repo from elsewhere in the GUI, just not by finding it in the list and checking the box.

The Journey to Discovery

Earlier this week, I had an issue deploying Firefox 60.1esr with the desired settings for our Labs. The issue was due to changes in how Firefox allows you to manage settings with its new Quantum platform. I had been using Greg Neagle’s FirefoxAutoconfig.pkg recipe, which is a great example of how you can use AutoPkg to create a custom installer package by combining what you download with something you have prepared separately. Since I was on the ESR track, there should have been a set of steps I could have followed to keep using FirefoxAutoconfig until the next major ESR release, but apparently I did not follow them. Rather than spend any more time on a dying methodology, I decide to bite the bullet and migrate my settings to the policies.json method that was supported on the new platform. I wrote that file, tested it, and was now ready to automate its inclusion in my deployment. I had heard mention in the MacAdmins Slack of someone writing a recipe for this new method, so I went searching.

Specifically, since I had been trying to debug the problem with my previous recipe, I had Terminal open, so it was simple enough to type in autopkg search firefox. I found that both mosen and Neil Martin had written recipes that used Greg’s FirefoxAutoconfig.pkg recipe as a starting point and then adapted it to the new policies.json method. Since I only needed a .pkg recipe, I could choose either version (mosen wrote a .munki recipe as well). For those who attended (or subsequently watched) that aforementioned presentation at Penn State, you may remember that I discussed some pros and cons of declaring certain values as Input as compared to hard coding them. Neil’s approach was more similar to the way I would have written this recipe, so I chose his. That’s when the fun began.

Sure, I could have added Neil’s repo at the command line, but I’m a bad typist, so I decided to jump into AutoPkgr, find Neil’s recipe repo, and check a box. Whoops! The repo wasn’t there. But I knew that it existed because AutoPkg just found a recipe from it. Since I was already in AutoPkgr, I decided I would just type in the URL in the “Add a repo URL manually” box (so much for saving typing!) and click Add. That worked perfectly. I was then able to do what I needed for the rest of my current workflow to get my customized Firefox installer package assembled and loaded into my current deployment system. Crisis averted.

Fast forward to Friday. The panic is over; my Fall Redeployment is basically done. I see a note in the #firefox channel that someone else had a bit of trouble finding that same recipe. So I decide to do some further digging to see if this was a bigger problem. At work, I have AutoPkg running on my daily driver, but only for maintaining the software on that Mac. I have a separate Mac mini that runs AutoPkg and AutoDMG to support my Lab deployment and OS testing needs. I had found the problem on the latter, but had not checked elsewhere. It turns out that Neil’s repo didn’t show up on my personal work Mac either. For fun, I decided to count the repos listed in that GUI window in AutoPkgr. It was roughly 100. I compared that to the recipe repo count on GitHub: 103. So I posted a comment on Slack in both #firefox and #autopkg channels seeking insights from others. I got useful responses from @elios (as usual) and Elliot Jordan.

It seems the likely culprit is GitHub’s API results limit for the type of request AutoPkgr is making. If you’ve tried executing autopkg search with a vague or common search term, you may have seen the message: Warning: Search yielded more than 100 results. Please try a more specific search term. (If you make the equivalent GitHub Search request in AutoPkgr, you’ll still be capped at 100 results, but you won’t be notified about the truncated results.) That seems to be the same for AutoPkgr’s request for the list of recipe repos.

For reference, here are the AutoPkg project recipe repos that are not listed by default by AutoPkgr (updated 2019-09-21):

Surprisingly, this seems to have been the case since 2017 November 07, as these repos were added on or after that date. Of course, any recipe repos added from now on to the project will have the same problem.

So I’m documenting this here simply to set expectations. AutoPkgr isn’t going to list all the available recipe repos in its interface.[1] You can still add the above repos from within AutoPkgr (by typing their full git path manually, or by doing a “Search for a recipe on GitHub” for a recipe you know to be in the desired repo and then add it from there) and you can always autopkg repo-add at the command line.

Additional recipe repos added to the list on 2019-01-19 and 2019-09-21.

[1] If you read the full discussion in the MacAdmins Slack #autopkg channel, you’ll know that there may be a workaround that doesn’t require a codebase change, but my feeble attempts to get a GitHub API Token working in that way have not had any success to this point. I’ll post an update if something comes to fruition.[Return to main text]