Development and maintenance

For development on the openeo package itself, it is recommended to install a local git checkout of the project in development mode (-e) with additional development related dependencies ([dev]) like this:

pip install -e .[dev]

Running the unit tests

The test suite of the openEO Python Client leverages the nice pytest framework. It is installed automatically when installing the openEO Python Client with the [dev] extra as shown above. Running the whole tests is as simple as executing:

pytest

There are a ton of command line options for fine-tuning (e.g. select a subset of tests, how results should be reported, …). Run pytest -h for a quick overview or check the pytest documentation for more information.

For example:

# Skip tests that are marked as slow
pytest -m "not slow"

Building the documentation

Building the documentation requires Sphinx and some plugins (which are installed automatically as part of the [dev] install).

Quick and easy

The easiest way to build the documentation is working from the docs folder and using the Makefile:

# From `docs` folder
make html

(assumes you have make available, if not: use python -msphinx -M html .  _build.)

This will generate the docs in HTML format under docs/_build/html/. Open the HTML files manually, or use Python’s built-in web server to host them locally, e.g.:

# From `docs` folder
python -m http.server 8000

Then, visit http://127.0.0.1:8000/_build/html/ in your browser

Like a Pro

When doing larger documentation work, it can be tedious to manually rebuild the docs and refresh your browser to check the result. Instead, use sphinx-autobuild to automatically rebuild on documentation changes and live-reload it in your browser. After installation (pip install sphinx-autobuild in your development environment), just run

# From project root
sphinx-autobuild docs/ --watch openeo/ docs/_build/html/

and then visit http://127.0.0.1:8000 . When you change (and save) documentation source files, your browser should now automatically refresh and show the newly build docs. Just like magic.

Creating a release

This section describes the procedure to create properly versioned releases of the openeo package that can be downloaded by end users (e.g. through pip from pypi.org) and depended on by other projects.

The releases will end up on:

Prerequisites

  • You have permissions to push branches and tags and maintain releases on the openeo-python-client project on GitHub.

  • You have permissions to upload releases to the openeo project on pypi.org

  • The Python virtual environment you work in has the latest versions of the twine package installed. If you plan to build the wheel yourself (instead of letting Jenkins do this), you also need recent enough versions of the setuptools and wheel packages.

Important files

setup.py

describes the metadata of the package, like package name openeo and version (which is extracted from openeo/_version.py).

openeo/_version.py

defines the version of the package. During general development, this version string should contain a pre-release segment (e.g. a1 for alpha releases, b1 for beta releases, etc) to avoid collision with final releases. For example:

__version__ = '0.4.7a1'

As discussed below, this pre-release suffix should only be removed during the release procedure and restored when bumping the version after the release procedure.

CHANGELOG.md

keeps track of important changes associated with each release. It follows the Keep a Changelog convention and should be properly updated with each bug fix, feature addition/removal, … under the Unreleased section during development.

Procedure

These are the steps to create and publish a new release of the openeo package. To be as concrete as possible, we will assume that we are about to release version 0.4.7.

  1. Make sure you are working on latest master branch, without uncommitted changes and all tests are properly passing.

  2. Create release commit:

    1. Drop the pre-release suffix from the version string in openeo/_version.py so that it just a “final” semantic versioning string, e.g. 0.4.7

    2. Update CHANGELOG.md: rename the “Unreleased” section title to contain version and date, e.g.:

      ## [0.4.7] - 2020-12-15
      

      remove empty subsections and start a new “Unreleased” section above it, like:

      ## [Unreleased]
      
      ### Added
      
      ### Changed
      
      ### Removed
      
      ### Fixed
      
    3. Commit these changes in git with a commit message like Release 0.4.7 and push to GitHub:

      git add openeo/_version.py CHANGELOG.md
      git commit -m 'Release 0.4.7'
      git push origin master
      
  3. Optional, but recommended: wait for VITO Jenkins to build this updated master (trigger it manually if necessary), so that a build of a final, non-alpha release 0.4.7 is properly uploaded to VITO artifactory.

  4. Create release on PyPI:

    1. Obtain a wheel archive of the package, with one of these approaches:

      • Path of least surprise: build wheel through GitHub Actions. Go to workflow “Build wheel”, manually trigger a build with “Run workflow” button, wait for it to finish successfully, download generated artifact.zip, and finally: unzip it to obtain openeo-0.4.7-py3-none-any.whl

      • If you know what you are doing and you’re sure you have a clean local checkout, you can also build it locally:

        python setup.py bdist_wheel
        

        This should create dist/openeo-0.4.7-py3-none-any.whl

    2. Upload this wheel to PyPI:

      python -m twine upload openeo-0.4.7-py3-none-any.whl
      
  5. Create a git version tag and push it to GitHub:

    git tag v0.4.7
    git push origin v0.4.7
    
  6. Create a release in GitHub: Go to https://github.com/Open-EO/openeo-python-client/releases/new, Enter v0.4.7 under “tag”, enter title: openEO Python Client v0.4.7, use the corresponding CHANGELOG.md section as description and publish it (no need to attach binaries).

  7. Bump version in openeo/_version.py, and append a pre-release “a1” suffix again, for example:

    __version__ = '0.4.8a1'
    

    Commit this (e.g. with message _version.py: next alpha version 0.4.8a1) and push to GitHub.

  8. Optionally: send a tweet about the release or announce it in the openEO Platform Forum .

Verification

The new release should now be available/listed at:

Here is a bash oneliner to verify that the PyPI release works properly:

(cd /tmp &&\
    python -m venv tmp-venv-openeo &&\
    . tmp-venv-openeo/bin/activate &&\
    pip install openeo==0.4.7 &&\
    python -c "import openeo;print(openeo);print(openeo.__version__)"\
)

It tries to install the package in a temporary virtual env, import it and print the package version.