Database

Renku uses an internal database store in the .renku/metadata that uses a custom implementation of the ZODB object database, with a separate file per main entity.

Custom database for store Persistent objects.

class renku.core.metadata.database.Cache

Database Cache.

clear()

Remove all entries.

get(oid, default=None)

See IPickleCache.

new_ghost(oid, object)

See IPickleCache.

pop(oid, default=<object object>)

Remove and return an object.

class renku.core.metadata.database.Database(storage)

The Metadata Object Database.

This class is equivalent to a persistent.DataManager and implements the persistent.interfaces.IPersistentDataManager interface.

add(object, oid=None)

Add a new object to the database.

NOTE: Normally, we add objects to indexes but this method adds objects directly to Dataset’s root. Use it only for singleton objects that have no Index defined for them (e.g. Project).

add_index(name, object_type, attribute=None, key_type=None)

Add an index.

add_root_object(name, obj)

Add an object to the DB root.

clear()

Remove all objects and clear all caches. Objects won’t be deleted in the storage.

commit()

Commit modified and new objects.

classmethod from_path(path)

Create a Storage and Database using the given path.

static generate_oid(object)

Generate an oid for a persistent.Persistent object based on its id.

get(oid)

Get the object by oid.

get_by_id(id)

Return an object by its id.

get_cached(oid)

Return an object if it is in the cache or will be committed.

static hash_id(id)

Return oid from id.

new_ghost(oid, object)

Create a new ghost object.

static new_oid()

Generate a random oid.

oldstate(object, tid)

See persistent.interfaces.IPersistentDataManager::oldstate.

readCurrent(object)

We don’t use this method but some Persistent logic require its existence.

register(object)

Register a persistent.Persistent object to be stored.

NOTE: When a persistent.Persistent object is changed it calls this method.

remove_from_cache(object)

Remove an object from cache.

remove_root_object(name)

Remove a root object from the database.

setstate(object)

Load the state for a ghost object.

class renku.core.metadata.database.Index(*args, **kwargs)

Database index.

Create an index where keys are extracted using attribute from an object or a key.

Parameters

• name – Index’s name

• object_type – Type of objects that the index points to

• attribute – Name of an attribute to be used to automatically generate a key (e.g. entity.path)

• key_type – Type of keys. If not None then a key must be provided when updating the index

add(object, *, key=None, key_object=None, verify=True)

Update index with object.

If Index._attribute is not None then key is automatically generated. Key is extracted from key_object if it is not None; otherwise, it’s extracted from object.

generate_key(object, *, key_object=None)

Return index key for an object.

Key is extracted from key_object if it is not None; otherwise, it’s extracted from object.

get(key, default=None)

Return an entry based on its key.

items()

Return an iterator of keys and values.

keys()

Return an iterator of keys.

property name

Return Index’s name.

property object_type

Return Index’s object_type.

pop(key, default=<object object>)

Remove and return an object.

values()

Return an iterator of values.

renku.core.metadata.database.MARKER = <object object>

NOTE: These are used as _p_serial to mark if an object was read from storage or is new

class renku.core.metadata.database.ObjectReader(database)

Deserialize objects loaded from storage.

deserialize(data)

Convert JSON to Persistent object.

set_ghost_state(object, data)

Set state of a Persistent ghost object.

class renku.core.metadata.database.ObjectWriter(database)

Serialize objects for storage in storage.

serialize(object)

Convert an object to JSON.

class renku.core.metadata.database.Persistent(*args, **kwargs)

Base Persistent class for renku classes.

Subclasses are assumed to be immutable once persisted to the database. If a class shouldn’t be immutable then subclass it directly from persistent.Persistent.

freeze()

Set immutable property.

property immutable

Return if object is immutable.

reassign_oid()

Reassign oid (after assigning a new identifier for example).

class renku.core.metadata.database.RenkuOOBTree(*args)

Customize BTrees.OOBTree.BTree implementation.

class renku.core.metadata.database.Storage(path)

Store Persistent objects on the disk.

load(filename)

Load data for object with object id oid.

store(filename, data, compress=False)

Store object.

renku.core.metadata.database.get_attribute(object, name)

Return an attribute of an object.

renku.core.metadata.database.get_class(type_name)

Return the class for a fully-qualified type name.

renku.core.metadata.database.get_type_name(object)

Return fully-qualified object’s type name.