Doing Python development under Mac OS
tl;dr? Jump right to the recipe
For reasons lost in the mists of time, I’ve been a Mac guy since 2013 - but I’ve been a command line (CLI) guy since 1990 (or earlier, hard to remember), and still am. A few years ago, I discovered brew, aka homebrew after finding out how out-of-date were so many of the standard UNIX/Linux tools included with Mac OS, e.g., bash.
Generalizing, the hardest part of working with the CLI on a Mac is having the same experience on all the platforms I use, which range from Windows 10 with WSL (some flavour of Ubuntu, cannot remember which), RHEL 7.6, Fedora 32, Mac OS 10.15, and Cygwin - and that problem persists across those other platforms, which is why I created a GitHub project to keep my dot files in snc.
Well. Until the last few days.
Since last week I’ve been using the most excellent Yamale to validate YAML schema I’ve defined for my SELinux Linear Assured Pipeline. I’ve even made some contributions to it - or, at least, am in the process of so doing…
…and that’s where things get weird.
Yamale depends upon Python and pip, the Python package manager, and upon tox, a test automation framework for Python - but standard Mac OS has neither
tox. OK, so
brew install pip and
brew install tox, right? Well, almost.
This is where things can get a little weird. Yamale’s
tox.ini, for example, depends upon two different versions of Python,
How does get multiple versions of Python? The short answer is pyenv, a python-independent shell-script based system for managing multiple Python environments. But that’s not the whole story.
This page intends itself as the correct answer - install
pyenv and go! - but it is lacking a step, as I discovered - and I discovered that page from this page, which tells us what we might do wrong, then what we should do to get things right - but a bootstrap problem remains….
The first page wants us to run
pip install pyenv - but Mac OS lacks
pip. The second page has a few brew-based recipes, but suggests that
brew is the wrong way to go, overall, and that we really want
pyenv. This third page tries to put things succinctly, but if you’re running
tox, there is YAS (Yet Another Step): By default,
tox works only with your default Python environment, whether that is standard or something installed via
tox to work with multiple Pythons out of the box, you need tox-pyenv - but that page mentions a lot of irrelevant
Circle CI stuff.
Before we go any further, I should note that whatever you do with brew - or with gem, if you get there - do NOT override system defaults - Mac OS and many utilities produced for the Mac depend upon them. Let brew manage things and install them cellar-only. You then need to find them via appropriate manipulation of PATH, CPPFLAGS, LDFLAGS, etc. Cf my dot file project for one way of doing this.
At any rate, none of those pages provide the complete answer. You know that you need
- Python (and possibly multiple versions of Python)
pip(not included with Mac OS)
tox, which depends upon
pyenv, which you can get a number of ways….
tox-pyenv, which depends upon
So what is one to do? Install
pyenv and use it to install
tox-pyenv several times over? That seems so….
I don’t know that what follows is a universally-accepted approach, but it is the shortest and cleanest path to where I needed to go (as verified by running
make clean; make in my
yamale top-level folder and having all
tox tests work out-of-the-box with both Python
3.8, no modifications to
yamale files required…).
Just The Recipe
# install brew $ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" # use brew to install python3, which comes with pip3 $ brew install python3 # let brew install this cask-only # figure out your path to pip3, installed with python3 $ <path to pip3> install tox # install py-env using brew $ brew install py-env $ <path to pip3> install tox-pyenv
Once you’ve done that, figure out what versions of Python your particular
tox.ini specifies; for
yamale, these are
3.8 which I’ve mentioned a few times already…. Assuming these are what you need, the rest of your recipe is as follows:
$ brew install pyenv $ pyenv install 3.6 # lists multiple versions, choose appropriate minor #, e.g., $ pyenv install 3.6.11 $ pyenv install 3.8 # lists multiple versions, choose appropriate minor #, e.g., $ pyenv install 3.8.5 # almost there, now we want to use what we just installed $ cd <where Yamale lives> # have pyenv set the versions of python tox should use $ pyenv local 3.6.11 3.8.5 # that command creates a ./python-version file, which # is already in ./gitignore! Now confirm those.... $ pyenv version
In my case, the latter command shows
3.6.11 (set by /Users/pww/pipelineWork/yamale/.python-version) 3.8.5 (set by /Users/pww/pipelineWork/yamale/.python-version)
pyenv which shows me what I expected:
$ pyenv which python3.6 /Users/pww/.pyenv/versions/3.6.11/bin/python3.6 $ pyenv which python3.8 /Users/pww/.pyenv/versions/3.8.5/bin/python3.8
make clean; make and all should be well.
It was for me, at least.