Container

Container Auditor

class swift.container.auditor.ContainerAuditor(conf, logger=None)

Bases: swift.common.daemon.Daemon

Audit containers.

container_audit(path)

Audits the given container path

Parameters:path – the path to a container db
run_forever(*args, **kwargs)

Run the container audit until stopped.

run_once(*args, **kwargs)

Run the container audit once.

swift.container.auditor.random() → x in the interval [0, 1).

Container Backend

Pluggable Back-ends for Container Server

class swift.container.backend.ContainerBroker(db_file, timeout=25, logger=None, account=None, container=None, pending_timeout=None, stale_reads_ok=False)

Bases: swift.common.db.DatabaseBroker

Encapsulates working with a container database.

create_container_info_table(conn, put_timestamp, storage_policy_index)

Create the container_info table which is specific to the container DB. Not a part of Pluggable Back-ends, internal to the baseline code. Also creates the container_stat view.

Parameters:
  • conn – DB connection object
  • put_timestamp – put timestamp
  • storage_policy_index – storage policy index
create_object_table(conn)

Create the object table which is specific to the container DB. Not a part of Pluggable Back-ends, internal to the baseline code.

Parameters:conn – DB connection object
create_policy_stat_table(conn, storage_policy_index=0)

Create policy_stat table.

Parameters:
  • conn – DB connection object
  • storage_policy_index – the policy_index the container is being created with
db_contains_type = 'object'
db_reclaim_timestamp = 'created_at'
db_type = 'container'
delete_object(name, timestamp, storage_policy_index=0)

Mark an object deleted.

Parameters:
  • name – object name to be deleted
  • timestamp – timestamp when the object was marked as deleted
  • storage_policy_index – the storage policy index for the object
empty()

Check if container DB is empty.

Returns:True if the database has no active objects, False otherwise
get_db_version(conn)
get_info()

Get global data for the container.

Returns:dict with keys: account, container, created_at, put_timestamp, delete_timestamp, status_changed_at, object_count, bytes_used, reported_put_timestamp, reported_delete_timestamp, reported_object_count, reported_bytes_used, hash, id, x_container_sync_point1, x_container_sync_point2, and storage_policy_index.
get_info_is_deleted()

Get the is_deleted status and info for the container.

Returns:a tuple, in the form (info, is_deleted) info is a dict as returned by get_info and is_deleted is a boolean.
get_misplaced_since(start, count)

Get a list of objects which are in a storage policy different from the container’s storage policy.

Parameters:
  • start – last reconciler sync point
  • count – maximum number of entries to get
Returns:

list of dicts with keys: name, created_at, size, content_type, etag, storage_policy_index

get_policy_stats()
get_reconciler_sync()
has_multiple_policies()
list_objects_iter(limit, marker, end_marker, prefix, delimiter, path=None, storage_policy_index=0, reverse=False)

Get a list of objects sorted by name starting at marker onward, up to limit entries. Entries will begin with the prefix and will not have the delimiter after the prefix.

Parameters:
  • limit – maximum number of entries to get
  • marker – marker query
  • end_marker – end marker query
  • prefix – prefix query
  • delimiter – delimiter for query
  • path – if defined, will set the prefix and delimiter based on the path
  • storage_policy_index – storage policy index for query
  • reverse – reverse the result order.
Returns:

list of tuples of (name, created_at, size, content_type, etag)

make_tuple_for_pickle(record)
merge_items(item_list, source=None)

Merge items into the object table.

Parameters:
  • item_list – list of dictionaries of {‘name’, ‘created_at’, ‘size’, ‘content_type’, ‘etag’, ‘deleted’, ‘storage_policy_index’, ‘ctype_timestamp’, ‘meta_timestamp’}
  • source – if defined, update incoming_sync with the source
put_object(name, timestamp, size, content_type, etag, deleted=0, storage_policy_index=0, ctype_timestamp=None, meta_timestamp=None)

Creates an object in the DB with its metadata.

Parameters:
  • name – object name to be created
  • timestamp – timestamp of when the object was created
  • size – object size
  • content_type – object content-type
  • etag – object etag
  • deleted – if True, marks the object as deleted and sets the deleted_at timestamp to timestamp
  • storage_policy_index – the storage policy index for the object
  • ctype_timestamp – timestamp of when content_type was last updated
  • meta_timestamp – timestamp of when metadata was last updated
reported(put_timestamp, delete_timestamp, object_count, bytes_used)

Update reported stats, available with container’s get_info.

Parameters:
  • put_timestamp – put_timestamp to update
  • delete_timestamp – delete_timestamp to update
  • object_count – object_count to update
  • bytes_used – bytes_used to update
set_storage_policy_index(policy_index, timestamp=None)

Update the container_stat policy_index and status_changed_at.

set_x_container_sync_points(sync_point1, sync_point2)
storage_policy_index
update_reconciler_sync(point)
swift.container.backend.update_new_item_from_existing(new_item, existing)

Compare the data and meta related timestamps of a new object item with the timestamps of an existing object record, and update the new item with data and/or meta related attributes from the existing record if their timestamps are newer.

The multiple timestamps are encoded into a single string for storing in the ‘created_at’ column of the objects db table.

Parameters:
  • new_item – A dict of object update attributes
  • existing – A dict of existing object attributes
Returns:

True if any attributes of the new item dict were found to be newer than the existing and therefore not updated, otherwise False implying that the updated item is equal to the existing.

Container Server

Container Replicator

Container Sync

Container Updater

class swift.container.updater.ContainerUpdater(conf)

Bases: swift.common.daemon.Daemon

Update container information in account listings.

container_report(node, part, container, put_timestamp, delete_timestamp, count, bytes, storage_policy_index)

Report container info to an account server.

Parameters:
  • node – node dictionary from the account ring
  • part – partition the account is on
  • container – container name
  • put_timestamp – put timestamp
  • delete_timestamp – delete timestamp
  • count – object count in the container
  • bytes – bytes used in the container
  • storage_policy_index – the policy index for the container
container_sweep(path)

Walk the path looking for container DBs and process them.

Parameters:path – path to walk
get_account_ring()

Get the account ring. Load it if it hasn’t been yet.

get_paths()

Get paths to all of the partitions on each drive to be processed.

Returns:a list of paths
process_container(dbfile)

Process a container, and update the information in the account.

Parameters:dbfile – container DB to process
run_forever(*args, **kwargs)

Run the updater continuously.

run_once(*args, **kwargs)

Run the updater once.

swift.container.updater.random() → x in the interval [0, 1).

Table Of Contents

Previous topic

Account

Next topic

Account DB and Container DB

This Page