Plugin Support

Renku has several plugin hooks that can be used to add additional metadata and commands to the Renku CLI.

The following hooks are currently available:

Runtime Plugins

Runtime plugins are supported using the pluggy library.

Runtime plugins can be created as Python packages that contain the respective entry point definition in their setup.py file, like so:

from setuptools import setup

setup(
    ...
    entry_points={"renku": ["name_of_plugin = myproject.pluginmodule"]},
    ...
)

where myproject.pluginmodule points to a Renku hookimpl e.g.:

from renku.core.plugin import hookimpl

@hookimpl
def plugin_hook_implementation(param1, param2):
    ...

`renku run` hooks

Plugin hooks for renku run customization.

renku.core.plugin.run.activity_annotations(activity)[source]

Plugin Hook to add Annotation entry list to a Activity.

Run when creating an activity from a renku run.

Parameters:: activity – An Activity object to get annotations for.
Returns:: A list of renku.domain_model.provenance.annotation.Annotation objects.

renku.core.plugin.run.plan_annotations(plan)[source]

Plugin Hook to add Annotation entry list to a Plan.

Run when a Plan is created by renku run.

Parameters:: plan – A Plan object to get annotations for.
Returns:: A list of renku.domain_model.provenance.annotation.Annotation objects.

renku.core.plugin.run.pre_run(tool)[source]

Plugin Hook that gets called at the start of a renku run call.

Can be used to setup plugins that get executed during the run.

Parameters:: tool (PlanFactory) – The plan factory that captured the run.

This repository contains an implementation of an activity annotation plugin.

CLI Plugins

Command-line interface plugins are supported using the click-plugins <https://github.com/click-contrib/click-plugins> library.

As in case the runtime plugins, command-line plugins can be created as Python packages that contain the respective entry point definition in their setup.py file, like so:

from setuptools import setup

setup(
    ...
    entry_points={"renku.cli_plugins": ["mycmd = myproject.pluginmodule:mycmd"]},
    ...
)

where myproject.pluginmodule:mycmd points to a click command e.g.:

import click

@click.command()
def mycmd():
    ...

An example implementation of such plugin is available here.

Workflow Converter Plugins

Additional workflow converters can be implemented by extending renku.domain_model.workflow.converters.IWorkflowConverter. By default renku provides a CWL converter plugins that is used when exporting a workflow:

$ renku workflow export --format cwl <my_workflow>

class renku.domain_model.workflow.converters.IWorkflowConverter[source]

Abstract class for converting Plan to a workflow format.

abstract workflow_convert(workflow, basedir, output, output_format)[source]: Converts a single workflow step to a desired workflow format.

abstract workflow_format()[source]: Supported workflow description formats.

We created a dummy implementation of such a converter plugin.

Workflow Provider Plugins

Additional workflow providers can be implemented by extending renku.domain_model.workflow.provider.IWorkflowProvider. See Implementing a workflow provider for more information.

class renku.domain_model.workflow.provider.IWorkflowProvider[source]

Abstract class for executing Plan.

abstract workflow_execute(dag, basedir, config)[source]

Executes a given AbstractPlan using the provider.

Returns:: A list of output paths that were generated by this workflow.

abstract workflow_provider()[source]

Supported workflow description formats.

Returns:: A tuple of the provider itself and the workflow executor backends name.
Return type:: Tuple[IWorkflowProvider,str]