Module Reference¶
Interfaces¶
-
class
tooz.coordination.
CoordinationDriver
(member_id, parsed_url, options)[source]¶ -
CHARACTERISTICS
= ()¶ Tuple of
Characteristics
introspectable enum member(s) that can be used to interogate how this driver works.
-
__init__
(member_id, parsed_url, options)[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
__weakref__
¶ list of weak references to the object (if defined)
-
static
create_group
(group_id)[source]¶ Request the creation of a group asynchronously.
- Parameters
group_id (ascii bytes) – the id of the group to create
- Returns
None
- Return type
CoordAsyncResult
-
static
delete_group
(group_id)[source]¶ Delete a group asynchronously.
- Parameters
group_id (ascii bytes) – the id of the group to leave
- Returns
Result
- Return type
CoordAsyncResult
-
static
get_groups
()[source]¶ Return the list composed by all groups ids asynchronously.
- Returns
the list of all created group ids
- Return type
CoordAsyncResult
-
static
get_leader
(group_id)[source]¶ Return the leader for a group.
- Parameters
group_id – the id of the group:
- Returns
the leader
- Return type
CoordAsyncResult
-
static
get_lock
(name)[source]¶ Return a distributed lock.
This is a exclusive lock, a second call to acquire() will block or return False.
- Parameters
name – The lock name that is used to identify it across all nodes.
-
static
get_member_capabilities
(group_id, member_id)[source]¶ Return the capabilities of a member asynchronously.
- Parameters
group_id (ascii bytes) – the id of the group of the member
member_id (ascii bytes) – the id of the member
- Returns
capabilities of a member
- Return type
CoordAsyncResult
-
static
get_member_info
(group_id, member_id)[source]¶ Return the statistics and capabilities of a member asynchronously.
- Parameters
group_id (ascii bytes) – the id of the group of the member
member_id (ascii bytes) – the id of the member
- Returns
capabilities and statistics of a member
- Return type
CoordAsyncResult
-
static
get_members
(group_id)[source]¶ Return the set of all member ids of a group asynchronously.
- Returns
set of all member ids in the specified group
- Return type
CoordAsyncResult
-
static
heartbeat
()[source]¶ Update member status to indicate it is still alive.
Method to run once in a while to be sure that the member is not dead and is still an active member of a group.
- Returns
The number of seconds to wait before sending a new heartbeat.
-
static
join_group
(group_id, capabilities=b'')[source]¶ Join a group and establish group membership asynchronously.
- Parameters
group_id (ascii bytes) – the id of the group to join
capabilities (object) – the capabilities of the joined member
- Returns
None
- Return type
CoordAsyncResult
-
join_group_create
(group_id, capabilities=b'')[source]¶ Join a group and create it if necessary.
If the group cannot be joined because it does not exist, it is created before being joined.
This function will keep retrying until it can create the group and join it. Since nothing is transactional, it may have to retry several times if another member is creating/deleting the group at the same time.
- Parameters
group_id – Identifier of the group to join and create
capabilities – the capabilities of the joined member
-
join_partitioned_group
(group_id, weight=1, partitions=32)[source]¶ Join a group and get a partitioner.
A partitioner allows to distribute a bunch of objects across several members using a consistent hash ring. Each object gets assigned (at least) one member responsible for it. It’s then possible to check which object is owned by any member of the group.
This method also creates if necessary, and joins the group with the selected weight.
- Parameters
group_id – The group to create a partitioner for.
weight – The weight to use in the hashring for this node.
partitions – The number of partitions to create.
- Returns
A
Partitioner
object.
-
static
leave_group
(group_id)[source]¶ Leave a group asynchronously.
- Parameters
group_id (ascii bytes) – the id of the group to leave
- Returns
None
- Return type
CoordAsyncResult
-
leave_partitioned_group
(partitioner)[source]¶ Leave a partitioned group.
This leaves the partitioned group and stop the partitioner. :param group_id: The group to create a partitioner for.
-
requires_beating
= False¶ Usage requirement that if true requires that the
heartbeat()
be called periodically (at a given rate) to avoid locks, sessions and other from being automatically closed/discarded by the coordinators backing store.
-
static
run_elect_coordinator
()[source]¶ Try to leader elect this coordinator & activate hooks on success.
-
static
run_watchers
(timeout=None)[source]¶ Run the watchers callback.
This may also activate
run_elect_coordinator()
(depending on driver implementation).
-
static
stand_down_group_leader
(group_id)[source]¶ Stand down as the group leader if we are.
- Parameters
group_id – The group where we don’t want to be a leader anymore
-
start
(start_heart=False)[source]¶ Start the service engine.
If needed, the establishment of a connection to the servers is initiated.
-
stop
()[source]¶ Stop the service engine.
If needed, the connection to servers is closed and the client will disappear from all joined groups.
-
unwatch_elected_as_leader
(group_id, callback)[source]¶ Call a function when member gets elected as leader.
The callback functions will be executed when run_watchers is called.
- Parameters
group_id – The group id to watch
callback – The function to execute when a member leaves this group
-
unwatch_join_group
(group_id, callback)[source]¶ Stop executing a function when a group_id sees a new member joined.
- Parameters
group_id – The group id to unwatch
callback – The function that was executed when a member joined this group
-
unwatch_leave_group
(group_id, callback)[source]¶ Stop executing a function when a group_id sees a new member leaving.
- Parameters
group_id – The group id to unwatch
callback – The function that was executed when a member left this group
-
static
update_capabilities
(group_id, capabilities)[source]¶ Update member capabilities in the specified group.
- Parameters
group_id (ascii bytes) – the id of the group of the current member
capabilities (object) – the capabilities of the updated member
- Returns
None
- Return type
CoordAsyncResult
-
watch_elected_as_leader
(group_id, callback)[source]¶ Call a function when member gets elected as leader.
The callback functions will be executed when run_watchers is called.
- Parameters
group_id – The group id to watch
callback – The function to execute when a member leaves this group
-
Consul¶
Etcd¶
-
class
tooz.drivers.etcd.
EtcdDriver
(member_id, parsed_url, options)[source]¶ An etcd based driver.
This driver uses etcd provide the coordination driver semantics and required API(s).
The Etcd driver connection URI should look like:
etcd://[HOST[:PORT]][?OPTION1=VALUE1[&OPTION2=VALUE2[&...]]]
If not specified, HOST defaults to localhost and PORT defaults to 2379. Available options are:
Name
Default
protocol
http
timeout
30
lock_timeout
30
-
DEFAULT_HOST
= 'localhost'¶ Default hostname used when none is provided.
-
DEFAULT_PORT
= 2379¶ Default port used if none provided (4001 or 2379 are the common ones).
-
DEFAULT_TIMEOUT
= 30¶ Default socket/lock/member/leader timeout used when none is provided.
-
__init__
(member_id, parsed_url, options)[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
get_lock
(name)[source]¶ Return a distributed lock.
This is a exclusive lock, a second call to acquire() will block or return False.
- Parameters
name – The lock name that is used to identify it across all nodes.
-
heartbeat
()[source]¶ Update member status to indicate it is still alive.
Method to run once in a while to be sure that the member is not dead and is still an active member of a group.
- Returns
The number of seconds to wait before sending a new heartbeat.
-
lock_encoder_cls
¶ alias of
tooz.utils.Base64LockEncoder
-
unwatch_join_group
(group_id, callback)[source]¶ Stop executing a function when a group_id sees a new member joined.
- Parameters
group_id – The group id to unwatch
callback – The function that was executed when a member joined this group
-
unwatch_leave_group
(group_id, callback)[source]¶ Stop executing a function when a group_id sees a new member leaving.
- Parameters
group_id – The group id to unwatch
callback – The function that was executed when a member left this group
-
Etcd3gw¶
File¶
IPC¶
Memcached¶
-
class
tooz.drivers.memcached.
MemcachedDriver
(member_id, parsed_url, options)[source]¶ A memcached based driver.
This driver users memcached concepts to provide the coordination driver semantics and required API(s). It is fully functional and implements all of the coordination driver API(s). It stores data into memcache using expiries and msgpack encoded values.
The Memcached driver connection URI should look like:
memcached://[HOST[:PORT]][?OPTION1=VALUE1[&OPTION2=VALUE2[&...]]]
If not specified, HOST defaults to localhost and PORT defaults to 11211. Available options are:
Name
Default
timeout
30
membership_timeout
30
lock_timeout
30
leader_timeout
30
max_pool_size
None
General recommendations/usage considerations:
Memcache (without different backend technology) is a cache enough said.
-
CHARACTERISTICS
= (<Characteristics.DISTRIBUTED_ACROSS_THREADS: 'DISTRIBUTED_ACROSS_THREADS'>, <Characteristics.DISTRIBUTED_ACROSS_PROCESSES: 'DISTRIBUTED_ACROSS_PROCESSES'>, <Characteristics.DISTRIBUTED_ACROSS_HOSTS: 'DISTRIBUTED_ACROSS_HOSTS'>, <Characteristics.CAUSAL: 'CAUSAL'>)¶ Tuple of
Characteristics
introspectable enum member(s) that can be used to interogate how this driver works.
-
DEFAULT_TIMEOUT
= 30¶ Default socket/lock/member/leader timeout used when none is provided.
-
GROUP_LEADER_PREFIX
= b'_TOOZ_GROUP_LEADER_'¶ Key prefix attached to leaders of groups (used in name-spacing keys)
-
GROUP_LIST_KEY
= b'_TOOZ_GROUP_LIST'¶ Key where all groups ‘known’ are stored.
-
GROUP_PREFIX
= b'_TOOZ_GROUP_'¶ Key prefix attached to groups (used in name-spacing keys)
-
MEMBER_PREFIX
= b'_TOOZ_MEMBER_'¶ Key prefix attached to members of groups (used in name-spacing keys)
-
STILL_ALIVE
= b"It's alive!"¶ String used to keep a key/member alive (until it next expires).
-
__init__
(member_id, parsed_url, options)[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
create_group
(group_id)[source]¶ Request the creation of a group asynchronously.
- Parameters
group_id (ascii bytes) – the id of the group to create
- Returns
None
- Return type
CoordAsyncResult
-
delete_group
(group_id)[source]¶ Delete a group asynchronously.
- Parameters
group_id (ascii bytes) – the id of the group to leave
- Returns
Result
- Return type
CoordAsyncResult
-
get_groups
()[source]¶ Return the list composed by all groups ids asynchronously.
- Returns
the list of all created group ids
- Return type
CoordAsyncResult
-
get_leader
(group_id)[source]¶ Return the leader for a group.
- Parameters
group_id – the id of the group:
- Returns
the leader
- Return type
CoordAsyncResult
-
get_lock
(name)[source]¶ Return a distributed lock.
This is a exclusive lock, a second call to acquire() will block or return False.
- Parameters
name – The lock name that is used to identify it across all nodes.
-
get_member_capabilities
(group_id, member_id)[source]¶ Return the capabilities of a member asynchronously.
- Parameters
group_id (ascii bytes) – the id of the group of the member
member_id (ascii bytes) – the id of the member
- Returns
capabilities of a member
- Return type
CoordAsyncResult
-
get_members
(group_id)[source]¶ Return the set of all member ids of a group asynchronously.
- Returns
set of all member ids in the specified group
- Return type
CoordAsyncResult
-
heartbeat
()[source]¶ Update member status to indicate it is still alive.
Method to run once in a while to be sure that the member is not dead and is still an active member of a group.
- Returns
The number of seconds to wait before sending a new heartbeat.
-
join_group
(group_id, capabilities=b'')[source]¶ Join a group and establish group membership asynchronously.
- Parameters
group_id (ascii bytes) – the id of the group to join
capabilities (object) – the capabilities of the joined member
- Returns
None
- Return type
CoordAsyncResult
-
leave_group
(group_id)[source]¶ Leave a group asynchronously.
- Parameters
group_id (ascii bytes) – the id of the group to leave
- Returns
None
- Return type
CoordAsyncResult
-
run_watchers
(timeout=None)[source]¶ Run the watchers callback.
This may also activate
run_elect_coordinator()
(depending on driver implementation).
Mysql¶
PostgreSQL¶
Redis¶
Zake¶
Zookeeper¶
Exceptions¶
-
class
tooz.
ToozError
(message, cause=None)[source]¶ Exception raised when an internal error occurs.
Raised for instance in case of server internal error.
- Variables
cause – the cause of the exception being raised, when not none this will itself be an exception instance, this is useful for creating a chain of exceptions for versions of python where this is not yet implemented/supported natively.
-
__init__
(message, cause=None)[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
__weakref__
¶ list of weak references to the object (if defined)
-
class
tooz.coordination.
ToozConnectionError
(message, cause=None)[source]¶ Exception raised when the client cannot connect to the server.
-
class
tooz.coordination.
OperationTimedOut
(message, cause=None)[source]¶ Exception raised when an operation times out.
-
class
tooz.coordination.
GroupNotCreated
(group_id)[source]¶ Exception raised when the caller request an nonexistent group.
-
class
tooz.coordination.
GroupAlreadyExist
(group_id)[source]¶ Exception raised trying to create an already existing group.
-
class
tooz.coordination.
MemberAlreadyExist
(group_id, member_id)[source]¶ Exception raised trying to join a group already joined.
-
class
tooz.coordination.
MemberNotJoined
(group_id, member_id)[source]¶ Exception raised trying to access a member not in a group.
-
class
tooz.coordination.
GroupNotEmpty
(group_id)[source]¶ Exception raised when the caller try to delete a group with members.
-
tooz.utils.
raise_with_cause
(exc_cls, message, *args, **kwargs)[source]¶ Helper to raise + chain exceptions (when able) and associate a cause.
For internal usage only.
NOTE(harlowja): Since in py3.x exceptions can be chained (due to PEP 3134) we should try to raise the desired exception with the given cause.
- Parameters
exc_cls – the
ToozError
class to raise.message – the text/str message that will be passed to the exceptions constructor as its first positional argument.
args – any additional positional arguments to pass to the exceptions constructor.
kwargs – any additional keyword arguments to pass to the exceptions constructor.