I know this is not a priority and even if people are interested it doesn’t have to be done all at once, but it’s just an idea.

As we are starting to get quite a few things that run in Tox and I thought it might be nice to automate a few things to ensure we actually follow our style guide. One tool that I really like to use is pre-commit. It’s lightweight and runs fast and has hooks for tons of different languages and things. I think this could be a good addition to our toolbox and provide an easier way to run linting tasks. Obviously this does not replace tox for running tests, while in theory it could it would slow down each commit. It is really designed for linting and other checks. There are a ton of existing hooks for all sorts of checks for python, JS, yaml, Dockerfile, spelling mistakes, etc. You can also easily create your own hooks either locally or by creating a .pre-commit-hooks.yaml in a git repo. It can then be used in any repository by simply including it in the project’s config. For example adding the file to the repo could allow it to easily be included in repositories and run automatically via this file.

In order to use the plugin, each repo needs to have a config file called .pre-commit-config.yaml that contains the list of hooks to run. They will be run in the order they are listed. You can run it manually with pre-commit run --all-files but you can also run it automatically on every commit (it only runs on files added in the commit) if you install it as a pre-commit hook with pre-commit install. In order to allow for the configs to be shared with IDEs and other tools and provide consistent results, the configuration for said tools will be configured in the project’s pyproject.toml instead of the pre-commit config where possible.

Here is an example config that contains a few useful hooks.

.pre-commit-config.yaml

pyproject.toml

This example automatically:

• Removes trailing whitespaces (trailing-whitespace)

• Ensures a file is empty or ends with a newline (end-of-file-fixer)

• Validates any committed yaml (check-yaml) and toml (check-toml) files

• Upgrades old school python syntax to new syntax automatically using pyupgrade (eg. format/% to f-string, remove object inheritance, upgrade typing, etc). This profile limits it to python 3.7 as that’s what we currently have, but could be bumped to 3.9 for bullseye.

• Runs flake8 as it currently is run in tox

• Automatically sorts and groups imports with isort as our style guide suggests (in a way compatible with black).

• Automatically fixes styling issues with black

• Runs mypy to check static types to avoid issues (mypy can be left out of an initial phase for a project if adding types is out of scope)

Note: If you use pyupgrade on a repository that uses pydantic, FastAPI or other libraries where it will do runtime introspection of type hints, you might need to use the argument --keep-runtime-typing to ensure it does not automatically upgrade the types to newer syntax as it will cause errors when Python 3.9 tries to evaluate them

Obviously not all of these are required and some others might be useful, but it shows what it can do.

As we don’t want to duplicate this logic in tox probably we could remove the linting jobs from tox and run pre-commit in the CI as well do ensure we always have the expected behaviour.

If we want to install this for all repos, we can use the git init templatedir to automatically install it in all newly cloned repos, or we could add it to existing ones with find and pre-commit install .