Database¶
Renku uses an internal database store in the .renku/metadata
that uses a
custom implementation of the ZODB object database, with a separate file per
main entity.
Custom database for store Persistent objects.
- class renku.infrastructure.database.Cache[source]¶
Database
Cache
.- get(oid, default=None)[source]¶
See
IPickleCache
.- Parameters
oid – The oid of the object to get.
default – Default value to return if object wasn’t found (Default value = None).
- Returns
The object or default value if the object wasn’t found.
- pop(oid, default=<object object>)[source]¶
Remove and return an object.
- Parameters
oid – The oid of the object to remove from the cache.
default – Default value to return (Default value = MARKER).
- Raises
KeyError – If object wasn’t found and no default was given.
- Returns
The removed object or the default value if it doesn’t exist.
- class renku.infrastructure.database.Database(storage)[source]¶
The Metadata Object Database.
This class is equivalent to a
persistent.DataManager
and implements thepersistent.interfaces.IPersistentDataManager
interface.- add(object, oid=None)[source]¶
Add a new object to the database.
NOTE: Normally, we add objects to indexes but this method adds objects directly to Dataset’s root. Use it only for singleton objects that have no Index defined for them (e.g. Project).
- Parameters
object (persistent.Persistent) – The object to add.
oid (OID_TYPE, optional) – The oid for the object (Default value = None).
- add_index(name, object_type, attribute=None, key_type=None)[source]¶
Add an index.
- Parameters
- Returns
The created
Index
object.- Return type
- add_root_object(name, obj)[source]¶
Add an object to the DB root.
- Parameters
name (str) – The key of the object.
obj (Persistent) – The object to store.
- classmethod from_path(path)[source]¶
Create a Storage and Database using the given path.
- Parameters
path (Union[pathlib.Path, str]) – The path of the database.
- Returns
The database object.
- static generate_oid(object)[source]¶
Generate an
oid
for apersistent.Persistent
object based on its id.- Parameters
object (persistent.Persistent) – The object to create an oid for.
- Returns
An oid for the object.
- get(oid)[source]¶
Get the object by
oid
.- Parameters
oid (OID_TYPE) – The oid of the object to get.
- Returns
The object.
- Return type
persistent.Persistent
- get_by_id(id)[source]¶
Return an object by its id.
- Parameters
id (str) – The id to look up.
- Returns
The object with the given id.
- Return type
persistent.Persistent
- get_cached(oid)[source]¶
Return an object if it is in the cache or will be committed.
- Parameters
oid (OID_TYPE) – The id of the object to look up.
- Returns
The cached object.
- Return type
Optional[persistent.Persistent]
- static hash_id(id)[source]¶
Return
oid
from id.- Parameters
id (str) – The id to hash.
- Returns
The hashed id.
- Return type
OID_TYPE
- new_ghost(oid, object)[source]¶
Create a new ghost object.
- Parameters
oid (OID_TYPE) – The oid of the new ghost object.
object (persistent.Persistent) – The object to create a new ghost entry for.
- readCurrent(object)[source]¶
We don’t use this method but some Persistent logic require its existence.
- Parameters
object – The object to read.
- register(object)[source]¶
Register a persistent.Persistent object to be stored.
NOTE: When a persistent.Persistent object is changed it calls this method.
- Parameters
object (persistent.Persistent) – The object to register with the database.
- remove_from_cache(object)[source]¶
Remove an object from cache.
- Parameters
object (persistent.Persistent) – The object to remove.
- class renku.infrastructure.database.Index(*args, **kwargs)[source]¶
Database index.
Create an index where keys are extracted using attribute from an object or a key.
- Parameters
name (str) – Index’s name.
object_type – Type of objects that the index points to.
attribute (Optional[str], optional) – Name of an attribute to be used to automatically generate a key (e.g. entity.path).
key_type – Type of keys. If not None then a key must be provided when updating the index (Default value = None).
- add(object, *, key=None, key_object=None, verify=True)[source]¶
Update index with object.
If Index._attribute is not None then key is automatically generated. Key is extracted from key_object if it is not None; otherwise, it’s extracted from object.
- Parameters
object (persistent.Persistent) – Object to add.
key (Optional[str], optional) – Key to use in the index (Default value = None).
key_object – Object to use to extract a key from (Default value = None).
verify – Whether to check if the key is valid (Default value = True).
- generate_key(object, *, key_object=None)[source]¶
Return index key for an object.
Key is extracted from key_object if it is not None; otherwise, it’s extracted from object.
- Parameters
object (persistent.Persistent) – The object to generate a key for.
key_object – The object to derive a key from (Default value = None).
- Returns
A key for object.
- get(key, default=None)[source]¶
Return an entry based on its key.
- Parameters
key – The key of the entry to get.
default – Default value to return of entry wasn’t found (Default value = None).
- Returns
The found entry or the default value if it wasn’t found.
- property name¶
Return Index’s name.
- property object_type¶
Return Index’s object_type.
- renku.infrastructure.database.MARKER = <object object>¶
These are used as _p_serial to mark if an object was read from storage or is new
- class renku.infrastructure.database.ObjectReader(database)[source]¶
Deserialize objects loaded from storage.
- class renku.infrastructure.database.ObjectWriter(database)[source]¶
Serialize objects for storage in storage.
- class renku.infrastructure.database.RenkuOOBTree(*args)[source]¶
Customize
BTrees.OOBTree.BTree
implementation.
- class renku.infrastructure.database.Storage(path)[source]¶
Store Persistent objects on the disk.
- renku.infrastructure.database.get_attribute(object, name)[source]¶
Return an attribute of an object.