Simple steps to make your Python code better

Many of you have GIT code repositories, in this post I’ll show you how to make your Python code better.

I’ll use this repository as an example: https://github.com/Aykhan-sh/pandaseda

Fork it and try to make the code better.

Improving code readability

Improving the readability of your code is easy. We will use libraries for syntax formatting and validation.

First, let’s create configuration files for flake8, mypy and black in the repository.

Let’s install them first:

pip install black flake8 mypy

Take a look at flake8. Flake8 is a tool that allows you to scan a project’s code and detect stylistic errors and violations of various Python code conventions.

File setup.cfg  for flake8 and mypy

[flake8]
max-line-length = 120
exclude =.git,__pycache__,docs/source/conf.py,build,dist,tests
ignore = I101,D100,D101,D102,D103,D104,D105,D107,D401,E203,I900,N802,N806,N812,W503,S311,S605,S607,ISC003,ISC001,T101,T000,F541,PL123
per-file-ignores = __init__.py:F401,F403

[mypy]
ignore_missing_imports = True
disallow_untyped_defs = True
check_untyped_defs = True
warn_redundant_casts = True
no_implicit_optional = True
strict_optional = True

[mypy-tests.*]
ignore_errors = True

Parse the [flake8] block:

• max-line-length – maximum line length
• exclude – list of folders excluded from flake8 scanning
• ignore – a list of errors / warnings that are also excluded. For example: I101: The names in your from import are in the wrong order and D100 – Missing docstring
• per-file-ignores – exclude a specific file from scanning

flake8 is very easy to run:

flake8

Remember that flake8 does not modify the code, it will simply test it. Errors will have to be corrected manually.

Now let’s talk about mypy. Python does not have mandatory static typing, but it is recommended to add types to function arguments and return types. To do this, just run mypy and don’t forget to correct the errors:

mypy.

Create a pyproject.toml file for black. black will help you format your code according to the standard.

File pyproject.toml

[tool.black]
line-length = 119
target-version = ['py36']
include = '\.pyi?$'
exclude = '''
/(
    \.eggs
  | \.git
  | \.hg
  | \.mypy_cache
  | \.tox
  | \.venv
  | _build
  | buck-out
  | build
  | dist
)/
'''

Analyze the code:

• line-length – line length
• target-version – Python versions. py36 – Python 3.6, it is possible for other versions as well.
• include – a list of what to include in the formatting
• exclude – a list of what to exclude from formatting

The launch is also very simple:

black.

I recommend saving the edited file. I have also fixed all mypy errors. Let’s take a look at what has changed.

This is what the code looked like before it was edited:

And so after editing:

The code has become more readable and now we have static typing. Pay particular attention to the Union type. It is necessary to use it when it is allowed to use not all types, but only some. You need to list them in square brackets.

isort

isort is a Python library for sorting imports alphabetically, with automatic division into sections and by type.

Installation:

pip install isort

Usage:

isort.

This is how our imports looked before editing:

And so after applying isort:

I also recommend saving the edited file.

pre-commit hook

We can run black, flake8 and mypy manually, but this is not convenient. We can automate the process using a pre-commit hook.

File .pre-commit-config.yaml

repos:
  - repo: https://github.com/asottile/pyupgrade
    rev: v2.19.4
    hooks:
      - id: pyupgrade
        args: [ "--py38-plus" ]
  - repo: https://github.com/pre-commit/mirrors-isort
    rev: 1ba6bfc # Use the revision sha / tag you want to point at
    hooks:
      - id: isort
        args: ["--profile", "black"]
  - repo: https://github.com/psf/black
    rev: 21.6b0
    hooks:
      - id: black
  - repo: https://gitlab.com/pycqa/flake8
    rev: 3.9.2
    hooks:
      - id: flake8
        language_version: python3
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.0.1
    hooks:
      - id: check-docstring-first
      - id: check-json
      - id: check-merge-conflict
      - id: check-yaml
      - id: debug-statements
      - id: end-of-file-fixer
      - id: trailing-whitespace
      - id: requirements-txt-fixer
  - repo: https://github.com/pre-commit/mirrors-pylint
    rev: 56b3cb4
    hooks:
      - id: pylint
        args:
          - --max-line-length=120
          - --ignore-imports=yes
          - -d duplicate-code
  - repo: https://github.com/pre-commit/pygrep-hooks
    rev: v1.9.0
    hooks:
      - id: python-check-mock-methods
      - id: python-use-type-annotations
      - id: python-check-blanket-noqa
      - id: python-use-type-annotations
      - id: text-unicode-replacement-char
  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: 9feadeb
    hooks:
      - id: mypy
        exclude: ^tests/
        args:
          [
              --disallow-untyped-defs,
              --check-untyped-defs,
              --warn-redundant-casts,
              --no-implicit-optional,
              --strict-optional
          ]

Now each commit will trigger our formatting and validation. Now let’s make Github run them on every pull request.

Github Actions

For our purposes, we will use Github Action. Let’s create a ci.yaml file

File .github/workflows/ci.yaml

name: Python package

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]
jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [3.6]
    steps:
    - uses: actions/checkout@v2
    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}
    # You can test your matrix by printing the current Python version
    - name: Display Python version
      run: python -c "import sys; print(sys.version)"
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements_dev.txt
    - name: Run black
      run:
        black --check .
    - name: Run flake8
      run: flake8
    - name: Run Mypy
      run: mypy pandaseda

Pay particular attention to:

- name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements_dev.txt
    - name: Run black
      run:
        black --check .
    - name: Run flake8
      run: flake8
    - name: Run Mypy
      run: mypy pandaseda

Here are our checks. Run Black here with the check flag. Black will not edit the code, but it will check. That is why it is better to save the file after running black beforehand.

Action is now ready, you can run it on any commit. After launch, you can see the progress and the result in the Action tab.

In case of errors, an email will be sent to the mail:

The error itself can also be viewed in Action:

Conclusion

In this post, I managed to consider several libraries for working with your code. We looked at the pre-commit hook and Github Actions separately.

Additional material

• GitHub Actions Documentation
• isort
• mypy documentation
• black

Read more my posts