Renku Workflow¶
Renku uses PROV-O and its own Renku ontology to represent workflows.
Plans¶
Represent run templates.
- class renku.core.models.workflow.plan.AbstractPlan(*args, **kwargs)[source]¶
Abstract base class for all plans.
- class renku.core.models.workflow.plan.Plan(*args, **kwargs)[source]¶
Represent a renku run execution template.
- assign_new_id()[source]¶
Assign a new UUID.
This is required only when there is another plan which is exactly the same except the parameters’ list.
- copy()[source]¶
Create a copy of this plan.
Required where a plan is used several times in a workflow but we need to set different values on them.
- is_similar_to(other)[source]¶
Return true if plan has the same inputs/outputs/arguments as another plan.
- property keywords_csv¶
Comma-separated list of keywords associated with workflow.
- class renku.core.models.workflow.plan.PlanDetailsJson(*, only=None, exclude=(), many=False, context=None, load_only=(), dump_only=(), partial=False, unknown=None)[source]¶
Serialize a plan to a response object.
Represent a group of run templates.
- class renku.core.models.workflow.composite_plan.CompositePlan(*args, **kwargs)[source]¶
A plan containing child plans.
- find_link_by_target(target)[source]¶
Find a link on this or a child workflow that has target as a sink.
Parameters¶
Classes to represent inputs/outputs/parameters in a Plan.
- class renku.core.models.workflow.parameter.CommandInput(*, default_value=None, description=None, id, mapped_to=None, name=None, position=None, prefix=None, encoding_format=None, derived_from=None, postfix=None)[source]¶
An input to a command.
- class renku.core.models.workflow.parameter.CommandOutput(*, create_folder=False, default_value=None, description=None, id, mapped_to=None, name=None, position=None, prefix=None, encoding_format=None, derived_from=None, postfix=None)[source]¶
An output from a command.
- class renku.core.models.workflow.parameter.CommandParameter(*, default_value=None, description=None, id, name=None, position=None, prefix=None, derived_from=None, postfix=None)[source]¶
An argument to a command that is neither input nor output.
- class renku.core.models.workflow.parameter.CommandParameterBase(*, default_value, description, id, name, position=None, prefix=None, derived_from=None, postfix=None)[source]¶
Represents a parameter for a Plan.
- property actual_value¶
Get the actual value to be used for execution.
- property actual_value_set¶
Whether the actual_value on this parameter has been set at least once.
- property role¶
Return a unique role for this parameter within its Plan.
- class renku.core.models.workflow.parameter.MappedIOStream(*, id=None, stream_type)[source]¶
Represents an IO stream (stdin, stdout, stderr).
- class renku.core.models.workflow.parameter.ParameterLink(source, sinks, id)[source]¶
A link between a source and one or more sink parameters.
- class renku.core.models.workflow.parameter.ParameterMapping(*, default_value=None, description=None, id, name=None, mapped_parameters=None, **kwargs)[source]¶
A mapping of child parameter(s) to a parent CompositePlan.
- property actual_value¶
Get the actual value to be used for execution.
- property leaf_parameters¶
Return leaf (non-Mapping) parameters contained by this Mapping.
Renku Workflow Logic¶
Execution Graph¶
Build an execution graph for a workflow.
- class renku.core.management.workflow.concrete_execution_graph.ExecutionGraph(workflows, virtual_links=False)[source]¶
Represents an execution graph for one or more workflow steps.
- calculate_concrete_execution_graph(virtual_links=False)[source]¶
Create an execution DAG between Plans showing dependencies between them.
Resolve ParameterLink’s involving ParameterMapping’s to the underlying actual parameters and potentially also virtual links determined by parameter values.
- property cycles¶
Get potential cycles in execution graph.
- property workflow_graph¶
Return a subgraph with only workflows and their dependencies.
Value Resolution¶
Resolution of Workflow
execution values precedence.
- class renku.core.management.workflow.value_resolution.CompositePlanValueResolver(plan, values)[source]¶
Value resolution class for a
CompositePlan
.Applies values and default_values to a nested workflow.
Order of precedence is as follows (from lowest to highest): - Default value on a parameter - Default value on a mapping to the parameter - Value passed to a mapping to the parameter - Value passed to the parameter - Value propagated to a parameter from the source of a ParameterLink
- class renku.core.management.workflow.value_resolution.PlanValueResolver(plan, values)[source]¶
Value resolution class for a
Plan
.Applies values and default_values to a workflow.
Plan Factory¶
Used to create Plan
objects based on command line arguments
Represent a PlanFactory
for tracking workflows.
- class renku.core.management.workflow.plan_factory.PlanFactory(command_line, explicit_inputs=None, explicit_outputs=None, explicit_parameters=None, directory=None, working_dir=None, no_input_detection=False, no_output_detection=False, success_codes=None, stdin=None, stdout=None, stderr=None)[source]¶
Factory for creating a plan from a command line call.
- add_command_input(default_value, prefix=None, position=None, postfix=None, name=None, encoding_format=None)[source]¶
Create a CommandInput.
- add_command_output(default_value, prefix=None, position=None, postfix=None, encoding_format=None, name=None)[source]¶
Create a CommandOutput.
- add_command_output_from_parameter(parameter, name)[source]¶
Create a CommandOutput from a parameter.
- add_command_parameter(default_value, prefix=None, position=None, name=None)[source]¶
Create a CommandParameter.
- get_stream_mapping_for_value(value)[source]¶
Return a stream mapping if value is a path mapped to a stream.
- renku.core.management.workflow.plan_factory.add_indirect_parameter(working_dir, name, value)[source]¶
Add a parameter to indirect parameters.
- renku.core.management.workflow.plan_factory.delete_indirect_files_list(working_dir)[source]¶
Remove indirect inputs, outputs, and parameters list.
- renku.core.management.workflow.plan_factory.get_indirect_inputs_path(client_path)[source]¶
Return path to file that contains indirect inputs list.
- renku.core.management.workflow.plan_factory.get_indirect_outputs_path(client_path)[source]¶
Return path to file that contains indirect outputs list.
Renku Provenance Logic¶
Use for building a direct acyclic graph of past executions.
Renku workflow commands.
- renku.core.management.workflow.activity.create_activity_graph(activities, remove_overridden_parents=True, with_inputs_outputs=False)[source]¶
Create a dependency DAG from activities.