Sharing of user-defined processes

Warning

Beta feature - At the time of this writing (July 2021), sharing of user-defined processes (publicly or among users) is not standardized in the openEO API. There are however some experimental sharing features in the openEO Python Client Library and some back-end providers that we are going to discuss here.

Be warned that the details of this feature are subject to change. For more status information, consult GitHub ticket Open-EO/openeo-api#310.

Publicly publishing a user-defined process.

As discussed in Building and storing user-defined process, user-defined processes can be stored with the save_user_defined_process() method on a on a back-end Connection. By default, these user-defined processes are private and only accessible by the user that saved it:

from openeo.processes import subtract, divide
from openeo.api.process import Parameter

# Build user-defined process
f = Parameter.number("f", description="Degrees Fahrenheit.")
fahrenheit_to_celsius = divide(x=subtract(x=f, y=32), y=1.8)

# Store user-defined process in openEO back-end.
udp = connection.save_user_defined_process(
    "fahrenheit_to_celsius",
    fahrenheit_to_celsius,
    parameters=[f]
)

Some back-ends, like the VITO/Terrascope back-end allow a user to flag a user-defined process as “public” so that other users can access its description and metadata:

udp = connection.save_user_defined_process(
    ...
    public=True
)

The sharable, public URL of this user-defined process is available from the metadata given by RESTUserDefinedProcess.describe. It’s listed as “canonical” link:

>>> udp.describe()
{
    "id": "fahrenheit_to_celsius",
    "links": [
        {
            "rel": "canonical",
            "href": "https://openeo.vito.be/openeo/1.0/processes/u:johndoe/fahrenheit_to_celsius",
            "title": "Public URL for user-defined process fahrenheit_to_celsius"
        }
    ],
    ...

Using a public UDP through URL based “namespace”

Some back-ends, like the VITO/Terrascope back-end, allow to use a public UDP through setting its public URL as the namespace property of the process graph node.

For example, based on the fahrenheit_to_celsius UDP created above, the “flat graph” representation of a process graph could look like this:

{
    ...
    "to_celsius": {
        "process_id": "fahrenheit_to_celsius",
        "namespace": "https://openeo.vito.be/openeo/1.0/processes/u:johndoe/fahrenheit_to_celsius",
        "arguments": {"f": 86}
    }

As a very basic illustration with the openEO Python Client library, we can create and evaluate a process graph, containing a fahrenheit_to_celsius call as single process, with Connection.datacube_from_process as follows:

cube = connection.datacube_from_process(
    process_id="fahrenheit_to_celsius",
    namespace="https://openeo.vito.be/openeo/1.0/processes/u:johndoe/fahrenheit_to_celsius",
    f=86
)
print(cube.execute())
# Prints: 30.0

Loading a published user-defined process as DataCube

From the public URL of the user-defined process, it is also possible for another user to construct, fully client-side, a new DataCube with Connection.datacube_from_json().

It is important to note that this approach is different from calling a user-defined process as described in Evaluate user-defined processes and Using a public UDP through URL based “namespace”. Connection.datacube_from_json() breaks open the encapsulation of the user-defined process and “unrolls” the process graph inside into a new DataCube. This also implies that parameters defined in the user-defined process have to be provided when calling Connection.datacube_from_json():

udp_url = "https://openeo.vito.be/openeo/1.0/processes/u:johndoe/fahrenheit_to_celsius"
cube = connection.datacube_from_json(
    udp_url,
    parameters={"f": 86},
)
print(cube.execute())
# Prints: 30.0

Note that Connection.datacube_from_json() not only supports loading UDPs from an URL but also from a raw JSON string or a local file path. For more information, also see Construct a DataCube from JSON.