API Reference

Complete reference of the public API.

Queues

Core implementation of a persistent AsyncIO queue.

class aiodiskqueue.queues.Queue

A persistent AsyncIO FIFO queue.

The content of a queue is stored on disk in a data file. This file only exists temporarily while there are items in the queue.

This class is not thread safe.

To create a new object the factory method create() must be used.

async classmethod create(data_path: str | Path, maxsize: int = 0, cls_storage_engine=None) → Queue

Create a new queue instance.

A data file will be created at the given path if it does not already exist.

If the data file already exists, if will be used to recreate the queue if possible. Should that fail, the existing data file will be backed up and the queue reset.

Please note that using two different instances with the same data file simultaneously is not recommended as it may lead to data corruption.

Parameters:

• data_path – Path of the data file for this queue. e.g. queue.dat

• maxsize – If maxsize is less than or equal to zero, the queue size is infinite. If it is an integer greater than 0, then put() blocks when the queue reaches maxsize until an item is removed by get().

• cls_storage_engine – Define the storage engine to be used. Default is DbmEngine.

empty() → bool

Return True if the queue is empty, False otherwise.

If empty() returns False it doesn’t guarantee that a subsequent call to get() will not raise QueueEmpty.

full() → bool

Return True if there are maxsize items in the queue.

If the queue was initialized with maxsize=0 (the default), then full() always returns False.

async get() → Any

Remove and return an item from the queue. If queue is empty, wait until an item is available.

async get_nowait() → Any

Remove and return an item if one is immediately available, else raise QueueEmpty.

async join() → None

Block until all items in the queue have been gotten and processed.

The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer calls task_done() to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero, join() unblocks.

property maxsize: int

Number of items allowed in the queue. 0 means unlimited.

async put(item: Any) → None

Put an item into the queue. If the queue is full, wait until a free slot is available before adding the item.

async put_nowait(item: Any) → None

Put an item into the queue without blocking.

If no free slot is immediately available, raise QueueFull.

Parameters:

item – Any Python object that can be pickled

qsize() → int

Return the approximate size of the queue. Note, qsize() > 0 doesn’t guarantee that a subsequent get() will not raise QueueEmpty.

async task_done() → None

Indicate that a formerly enqueued task is complete.

Used by queue consumers. For each get() used to fetch a task, a subsequent call to task_done() tells the queue that the processing on the task is complete.

If a join() is currently blocking, it will resume when all items have been processed (meaning that a task_done() call was received for every item that had been put() into the queue).

Raises ValueError if called more times than there were items placed in the queue.

Exceptions

Custom exceptions raised by this package.

exception aiodiskqueue.exceptions.QueueEmpty

Queue is empty.

exception aiodiskqueue.exceptions.QueueException

Top exception for exceptions raised by this package.

exception aiodiskqueue.exceptions.QueueFull

Queue is full.