Data Science, ML and Analytics Engineering

Simple steps to make your Python code better

Many of you have GIT code repositories, in this post I’ll show you how to make your Python code better.

I’ll use this repository as an example:

Fork it and try to make the code better.

Improving code readability

Improving the readability of your code is easy. We will use libraries for syntax formatting and validation.

First, let’s create configuration files for flake8, mypy and black in the repository.

Let’s install them first:

pip install black flake8 mypy

Take a look at flake8. Flake8 is a tool that allows you to scan a project’s code and detect stylistic errors and violations of various Python code conventions.

File setup.cfg  for flake8 and mypy

max-line-length = 120
exclude =.git,__pycache__,docs/source/,build,dist,tests
ignore = I101,D100,D101,D102,D103,D104,D105,D107,D401,E203,I900,N802,N806,N812,W503,S311,S605,S607,ISC003,ISC001,T101,T000,F541,PL123
per-file-ignores =,F403

ignore_missing_imports = True
disallow_untyped_defs = True
check_untyped_defs = True
warn_redundant_casts = True
no_implicit_optional = True
strict_optional = True

ignore_errors = True

Parse the [flake8] block:

  • max-line-length – maximum line length
  • exclude – list of folders excluded from flake8 scanning
  • ignore – a list of errors / warnings that are also excluded. For example: I101: The names in your from import are in the wrong order and D100 – Missing docstring
  • per-file-ignores – exclude a specific file from scanning

flake8 is very easy to run:


Remember that flake8 does not modify the code, it will simply test it. Errors will have to be corrected manually.

Now let’s talk about mypy. Python does not have mandatory static typing, but it is recommended to add types to function arguments and return types. To do this, just run mypy and don’t forget to correct the errors:


Create a pyproject.toml file for black. black will help you format your code according to the standard.

File pyproject.toml

line-length = 119
target-version = ['py36']
include = '\.pyi?$'
exclude = '''
  | \.git
  | \.hg
  | \.mypy_cache
  | \.tox
  | \.venv
  | _build
  | buck-out
  | build
  | dist

Analyze the code:

  • line-length – line length
  • target-version – Python versions. py36 – Python 3.6, it is possible for other versions as well.
  • include – a list of what to include in the formatting
  • exclude – a list of what to exclude from formatting

The launch is also very simple:


I recommend saving the edited file. I have also fixed all mypy errors. Let’s take a look at what has changed.

This is what the code looked like before it was edited:

Python code before black
Python code before black

And so after editing:

Python code after black
Python code after black

The code has become more readable and now we have static typing. Pay particular attention to the Union type. It is necessary to use it when it is allowed to use not all types, but only some. You need to list them in square brackets.


isort is a Python library for sorting imports alphabetically, with automatic division into sections and by type.


pip install isort



This is how our imports looked before editing:

imports before isort
imports before isort

And so after applying isort:

imports after isort
imports after isort

I also recommend saving the edited file.

pre-commit hook

We can run black, flake8 and mypy manually, but this is not convenient. We can automate the process using a pre-commit hook.

File .pre-commit-config.yaml

  - repo:
    rev: v2.19.4
      - id: pyupgrade
        args: [ "--py38-plus" ]
  - repo:
    rev: 1ba6bfc # Use the revision sha / tag you want to point at
      - id: isort
        args: ["--profile", "black"]
  - repo:
    rev: 21.6b0
      - id: black
  - repo:
    rev: 3.9.2
      - id: flake8
        language_version: python3
  - repo:
    rev: v4.0.1
      - id: check-docstring-first
      - id: check-json
      - id: check-merge-conflict
      - id: check-yaml
      - id: debug-statements
      - id: end-of-file-fixer
      - id: trailing-whitespace
      - id: requirements-txt-fixer
  - repo:
    rev: 56b3cb4
      - id: pylint
          - --max-line-length=120
          - --ignore-imports=yes
          - -d duplicate-code
  - repo:
    rev: v1.9.0
      - id: python-check-mock-methods
      - id: python-use-type-annotations
      - id: python-check-blanket-noqa
      - id: python-use-type-annotations
      - id: text-unicode-replacement-char
  - repo:
    rev: 9feadeb
      - id: mypy
        exclude: ^tests/

Now each commit will trigger our formatting and validation. Now let’s make Github run them on every pull request.

Github Actions

For our purposes, we will use Github Action. Let’s create a ci.yaml file

File .github/workflows/ci.yaml

name: Python package

    branches: [ master ]
    branches: [ master ]
    runs-on: ubuntu-latest
        python-version: [3.6]
    - uses: actions/checkout@v2
    - name: Set up Python
      uses: actions/setup-python@v2
        python-version: ${{ matrix.python-version }}
    # You can test your matrix by printing the current Python version
    - name: Display Python version
      run: python -c "import sys; print(sys.version)"
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements_dev.txt
    - name: Run black
        black --check .
    - name: Run flake8
      run: flake8
    - name: Run Mypy
      run: mypy pandaseda

Pay particular attention to:

- name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements_dev.txt
    - name: Run black
        black --check .
    - name: Run flake8
      run: flake8
    - name: Run Mypy
      run: mypy pandaseda

Here are our checks. Run Black here with the check flag. Black will not edit the code, but it will check. That is why it is better to save the file after running black beforehand.

Action is now ready, you can run it on any commit. After launch, you can see the progress and the result in the Action tab.

Github Action
Github Action

In case of errors, an email will be sent to the mail:

Github Action email
Github Action email

The error itself can also be viewed in Action:

Github Action Log
Github Action Log


In this post, I managed to consider several libraries for working with your code. We looked at the pre-commit hook and Github Actions separately.

Additional material

Read more my posts

Share it

If you liked the article - subscribe to my channel in the telegram or you can support me Become a Patron!

Other entries in this category: