Datasets

Models representing datasets.

class renku.domain_model.dataset.AnnotationJson(*, only=None, exclude=(), many=False, context=None, load_only=(), dump_only=(), partial=False, unknown=None)

Bases: Schema

Schema for Annotations.

class renku.domain_model.dataset.Dataset(*args, **kwargs)

Bases: Persistent

Represent a dataset.

add_or_update_files(files)

Add new files or update existing files.

clear_files()

Remove all files.

copy()

Return a clone of this dataset.

property creators_csv

Comma-separated list of creators associated with dataset.

property creators_full_csv

Comma-separated list of creators with full identity.

derive_from(dataset, creator, identifier=None, date_created=None)

Make self a derivative of dataset and update related fields.

property files

Return list of existing files.

find_file(path)

Find a file in the dataset using its relative path.

static generate_id(identifier)

Generate an identifier for Dataset.

get_datadir()

Return dataset’s data directory relative to project’s root.

is_derivation()

Return if a dataset has correct derived_from.

is_removed()

Return true if dataset is removed.

is_within_datadir(path)

Return True if a given path is inside dataset’s data directory.

property keywords_csv

Comma-separated list of keywords associated with dataset.

remove(date=None)

Mark the dataset as removed.

replace_identifier(identifier=None)

Replace dataset’s identifier and update relevant fields.

NOTE: Call this only for newly-created/-imported datasets that don’t have a mutability chain because it sets initial_identifier.

unlink_file(path, missing_ok=False)

Mark a file as removed using its relative path.

update_files_from(current_dataset, date=None)

Check current_files to reuse existing entries and mark removed files.

update_metadata(**kwargs)

Updates metadata.

update_metadata_from(other, exclude=None)

Update metadata from another dataset.

class renku.domain_model.dataset.DatasetCreatorsJson(*, only=None, exclude=(), many=False, context=None, load_only=(), dump_only=(), partial=False, unknown=None)

Bases: Schema

Schema for the dataset creators.

class renku.domain_model.dataset.DatasetDetailsJson(*, only=None, exclude=(), many=False, context=None, load_only=(), dump_only=(), partial=False, unknown=None)

Bases: Schema

Serialize a dataset to a response object.

static get_datadir(obj)

Get data directory.

class renku.domain_model.dataset.DatasetFile(*args, **kwargs)

Bases: Slots

A file in a dataset.

Create and return an empty instance of the class.

copy()

Return a clone of this object.

correct_linked_attribute()

Replace is_external attribute with linked for linked dataset files.

classmethod from_dataset_file(other)

Return a copy with a different id.

classmethod from_path(path, source=None, based_on=None, checksum=None, size=None)

Return an instance from a path.

static generate_id()

Generate an identifier for DatasetFile.

NOTE: ID should not rely on Entity properties because the same Entity can be added and removed multiple times. So, it should be marked by different DatasetFiles.

has_valid_checksum()

Return if file has a valid checksum.

has_valid_size()

Return if file has a valid size.

is_equal_to(other)

Compare content.

NOTE: id is generated randomly and should not be included in this comparison.

is_removed()

Return true if dataset is removed and should not be accessed.

remove(date=None)

Create a new instance and mark it as removed.

class renku.domain_model.dataset.DatasetFileDetailsJson(*, only=None, exclude=(), many=False, context=None, load_only=(), dump_only=(), partial=False, unknown=None)

Bases: Schema

Serialize dataset files to a response object.

class renku.domain_model.dataset.DatasetTag(*args, **kwargs)

Bases: Persistent

Represents a Tag of an instance of a dataset.

static generate_id(dataset_id, name)

Define default value for id field.

class renku.domain_model.dataset.ImageObjectJson(*, only=None, exclude=(), many=False, context=None, load_only=(), dump_only=(), partial=False, unknown=None)

Bases: Schema

ImageObject json schema.

class renku.domain_model.dataset.ImageObjectRequestJson(*, only=None, exclude=(), many=False, context=None, load_only=(), dump_only=(), partial=False, unknown=None)

Bases: Schema

ImageObject json schema.

class renku.domain_model.dataset.Language(*args, **kwargs)

Bases: Immutable

Represent a language of an object.

Create and return an empty instance of the class.

static generate_id(name)

Generate @id field.

class renku.domain_model.dataset.RemoteEntity(*args, **kwargs)

Bases: Slots

Reference to an Entity in a remote repository.

Create and return an empty instance of the class.

static generate_id(checksum, path, url)

Generate an id.

class renku.domain_model.dataset.Url(*, id=None, url=None, url_str=None, url_id=None)

Bases: object

Represents a schema URL reference.

static generate_id(url_str, url_id)

Generate an identifier for Url.

property value

Returns the url value as string.

renku.domain_model.dataset.generate_default_slug(name, version=None)

Get dataset slug.

renku.domain_model.dataset.get_file_path_in_dataset(dataset, dataset_file)

Return path of a file relative to dataset’s data dir.

renku.domain_model.dataset.is_dataset_slug_valid(slug)

Check if a given slug is valid.