API reference¶
Everything below is importable from drf_batch unless noted. The
pipeline, conflicts and writers modules are internal and may change
without notice before 1.0.
Viewset mixin¶
BatchMixin ¶
Batch create/update/destroy endpoint for a viewset.
Attributes:
batch_serializer_class: per-row serializer; falls back to the
viewset's serializer_class.
batch_collector_class: optional full-replacement collection hook.
None (default) auto-collects the serializer's batch fields.
batch_max_size: per-view override of the BATCH_MAX_SIZE setting.
batch_use_bulk: bulk writes (default) instead of per-row
create / update / delete() (which preserve save()
overrides and model signals).
batch_bulk_size: chunk size for the bulk writes; None (default)
uses the BULK_SIZE setting.
batch_actions: operations the endpoint enables ("create",
"update", "replace", "destroy"); the others are a
405. "replace" (PUT) is not enabled by default.
batch_conflict_status: HTTP status for conflicts (default 409).
batch_retry_policy: a :class:~drf_batch.retry.RetryPolicy instance
or class, or None (run the write once).
batch_lock_targets: lock update/destroy targets with
select_for_update() during the write (default True).
batch_concurrent_collection: fetch the batch fields concurrently
as a dependency graph (default True); False fetches
them one by one, sync fetches on the request thread.
batch_response: "count" (default) returns {"count": N};
"data" returns the serialized rows (create/update) or the
deleted pks (destroy).
batch_create ¶
POST /<resource>/batch/: create rows (201).
batch_update ¶
PATCH /<resource>/batch/: partial update by primary key (200).
batch_replace ¶
PUT /<resource>/batch/: full update by primary key (200).
batch_destroy ¶
DELETE /<resource>/batch/: delete by primary key (200).
run_batch ¶
Validate and write one batch: the body of the three actions.
perform_batch_create ¶
Write a validated create batch; override to pass save() kwargs.
perform_batch_update ¶
Write a validated update batch; override to pass save() kwargs.
perform_batch_replace ¶
Write a validated PUT batch; defaults to :meth:perform_batch_update.
perform_batch_destroy ¶
Delete a validated batch of targets.
get_batch_response ¶
The success body. batch_response picks count or data.
get_batch_operation ¶
The batch operation of the current action, or None.
get_serializer_class ¶
The batch serializer class for batch actions, else DRF's default.
get_batch_serializer_class ¶
batch_serializer_class, falling back to serializer_class.
get_batch_serializer ¶
The :class:~drf_batch.BatchListSerializer for one batch.
get_batch_config ¶
The :class:~drf_batch.BatchConfig for one request.
Update/destroy get get_queryset() (scoping) and every target is
checked with :meth:check_batch_object_permissions.
check_batch_object_permissions ¶
Object-permission check for one update/destroy target.
get_batch_context ¶
The serializer context (get_serializer_context()).
get_batch_rows ¶
The payload rows; anything but a JSON array is a 400.
get_batch_bulk_size ¶
batch_bulk_size, else the BULK_SIZE setting.
get_batch_partial ¶
Whether update rows are partial (False for batch_replace).
get_batch_conflict_status ¶
The status of the conflict pass (batch_conflict_status).
get_batch_retry_policy ¶
batch_retry_policy, instantiated if it is a class.
get_batch_collector ¶
The full-replacement collection hook instance, or None.
enforce_batch_size ¶
Raise :class:~drf_batch.BatchSizeLimitExceeded (413) if too large.
Serializers¶
BatchSerializer ¶
Bases: ModelSerializer
Per-row serializer for batch payloads.
The maps produced by the request's collection (auto-collected from the
declared batch fields, or returned by a replacement
:class:~drf_batch.collectors.BatchCollector) are available as
self.collected (a plain dict; {} when nothing was collected), so
batch fields and user validate() methods can resolve lookups with
zero I/O. :meth:fetched returns one batch field's value for the row
being validated, including fields the row did not send (read_only,
derived or stored keys).
Meta options:
batch_unique_together(default()): extra cross-row/uniqueness rules for the 409 conflict pass — each a field name (str) or a composite (tuple[str, ...]) of concrete fields onMeta.model.batch_model_unique(defaultTrue): also derive rules from the model'suniquefields,unique_togetherand unconditionalUniqueConstraints (replacing DRF's per-row unique validators).
fetched ¶
The collected value of batch field name for the current row.
For validate() / validate_<field>(), in a batch or a
single-object validation. The key is the one the row sent, else the
field's derived (derived_from) or stored (update) key. None
when the row has no key for the field, the fetch did not return it,
or outside validation. A key the row
did not send is never a "not found" error: check for None here.
A writable :class:~drf_batch.BatchRelatedField the row omits has
a stored key only when another batch field reads it
(derived_from / depends_on); use self.instance otherwise.
get_batch_fields ¶
All declared :class:BatchField instances, in field order.
many_init
classmethod
¶
many=True -> :class:BatchListSerializer.
Accepts batch_config= (a :class:~drf_batch.BatchConfig) and the
config's fields as keyword shortcuts (operation=, ...).
BatchListSerializer ¶
Bases: ListSerializer
List serializer with index-aligned errors and atomic writes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config
|
BatchConfig | None
|
the :class: Any :class: Bulk create/update is flat-fields-only: every writable field must be a concrete model field (checked up front). |
None
|
save ¶
Run the all-or-none write; kwargs are merged into every row.
For destroy instance is the list of deleted pks.
Configuration¶
BatchConfig
dataclass
¶
Options for one batch run.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
operation
|
BatchOperation
|
create / update / destroy (a :class: |
CREATE
|
use_bulk
|
bool
|
|
True
|
bulk_size
|
int | None
|
chunk size for the bulk writes; |
None
|
collector
|
BatchCollector | None
|
optional full-replacement collection hook. |
None
|
conflict_status
|
int
|
HTTP status of the conflict pass (default 409). |
HTTP_409_CONFLICT
|
retry_policy
|
RetryPolicy | None
|
wraps the write; |
None
|
queryset
|
QuerySet[Any] | None
|
the queryset update/destroy targets are resolved from.
:class: |
None
|
check_object
|
Callable[[Any], None] | None
|
called with every update/destroy target; raising
|
None
|
lock_targets
|
bool
|
re-fetch update/destroy targets with
|
True
|
concurrent_collection
|
bool
|
run the batch fields' fetches as a
dependency graph on the event loop, each one as soon as its
|
True
|
partial
|
bool
|
update rows are partial ( |
True
|
BatchOperation ¶
Bases: StrEnum
The three batch operations, served on one /batch/ route.
Batch fields¶
BatchField ¶
Bases: Field
Base class for fields resolved against the request's collected maps.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
map
|
str | None
|
key into the request's |
None
|
error
|
str | None
|
optional template for the miss message; |
None
|
depends_on
|
Any
|
names of other batch fields whose fetches must
finish first. Fetches without a dependency between them run
concurrently; every fetch receives a snapshot of the maps collected
so far, which always includes its |
()
|
derived_from
|
str | tuple[str, ...] | None
|
a field name or a tuple of names the key is built
from when the row does not send this field (always, for a
|
None
|
key
|
Callable[..., Any] | None
|
builds the key from the |
None
|
prepare_key ¶
Coerce the raw row value to a lookup key.
Must be idempotent. None (missing or null in the row) maps to
the :data:NO_KEY sentinel: the field contributes no key, and
required/blank handling happens during row validation.
fetch ¶
Return the {lookup_key: value} map for keys.
collected is a snapshot of the maps finished so far (always
including this field's depends_on); context is the
serializer context. May be a sync method (run on a worker thread,
or on the request thread inside a transaction) or an async def
(run on the event loop).
derived_key ¶
The key built from the derived_from values (:data:NO_KEY if none).
stored_key_path ¶
Where an update target stores this field's key.
(attributes, select_related): the attribute chain to read on the
target, and the relation the target query must join for it (or
None). None when the field's source is not a concrete field
of model. A ForeignKey/OneToOneField reads its column
(ticker_id).
to_representation ¶
Return the lookup key that maps to value in the collected map.
The reverse map is built once per request and cached in the serializer context next to the mapping it was built from (so it can never outlive or mismatch that mapping). Outside a batch request (nothing collected) the value is returned unchanged. Duplicate map entries pointing at the same value: last key wins.
BatchRelatedField ¶
Bases: BatchField
A relation field auto-collected from the database.
The model is auto-derived from the ForeignKey (or OneToOneField) on
Meta.model named by the field's source (the field name by
default; DRF reads the input row key from the field name and places the
validated value under the source); pass model= explicitly for
non-FK lookups. Row values are coerced to the lookup field's type
(lookup= defaults to pk); a malformed value is a row error, not
a 500.
Like DRF's RelatedField, the candidate rows can be scoped: pass
queryset= or override :meth:get_queryset (self.context holds the
request) so a client cannot reference rows it may not see. The default is
every row of the model's default manager.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
map
|
str | None
|
key into the collected maps; defaults to the field name. |
None
|
queryset
|
Any
|
rows the lookup may resolve to; its model is the lookup model. |
None
|
model
|
Any
|
lookup model; defaults to |
None
|
lookup
|
str | None
|
field on the lookup model to match against; defaults to
|
None
|
error
|
str | None
|
miss-message template ( |
None
|
depends_on
|
Any
|
batch fields that must be fetched first. |
()
|
derived_from
|
str | tuple[str, ...] | None
|
see :class: On update, a row that omits a writable related field contributes the
target's stored key only when another batch field reads this one
( |
None
|
BatchServiceField ¶
Bases: BatchField
A batch field whose map comes from a user fetch callable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
map
|
str | None
|
key into the collected maps; defaults to the field name. |
None
|
fetch
|
Any
|
a sync or |
None
|
error
|
str | None
|
miss-message template ( |
None
|
depends_on
|
Any
|
batch fields that must be fetched first. |
()
|
derived_from
|
str | tuple[str, ...] | None
|
see :class: On update, a row that omits the field contributes the target's stored value of the field's model attribute (column, or FK column), so the service sees a key for every row without any opt-in.
The fetch is invoked even when no row supplied a key for this field
(unlike a database lookup, which is skipped for an empty key set), since
its map may be derived from the other collected maps; return |
None
|
Collector¶
BatchCollector ¶
Base class for a full-replacement collection hook.
Attributes:
collect_timeout: per-collector timeout in seconds for an async
collect; None (default) falls back to the
COLLECT_TIMEOUT setting. Ignored for sync collectors.
collect ¶
collect(
keys: dict[str, set[Any]], context: BatchContext
) -> dict[str, Any] | Awaitable[dict[str, Any]]
Resolve the coerced key sets into the maps validation will use.
keys is the {map_name: {coerced_key, ...}} derived from the
serializer's batch fields. Return a {map_name: {lookup_key:
value}} mapping — every map the serializer's batch fields
need (a missing map means every row gets a miss error for it). The
result is exposed to serializers as serializer.collected and as
context["collected"].
May be overridden as async def collect(...) for concurrent
lookups (the awaitable is then bridged to the shared loop); see the
module docstring for the threading model.
to_thread
async
¶
Run func(*args, **kwargs) on the default thread pool.
Use inside an async collect for blocking calls (notably Django
ORM queries). The call runs on the worker's own connection, managed
like a request's (CONN_MAX_AGE applies; see
:func:drf_batch.loop.to_thread), so it does not see the request's
uncommitted writes: use :meth:run_sync for that.
run_sync
async
¶
Run func(*args, **kwargs) where it sees the request's database.
When the request's connection is in a transaction
(ATOMIC_REQUESTS, an outer atomic(), a TestCase), the
call runs on the request thread and sees its uncommitted writes;
otherwise on a worker thread, like :meth:to_thread. See
:func:drf_batch.loop.run_sync.
taskgroup ¶
An asyncio.TaskGroup for structured concurrency inside collect.
BatchContext ¶
Bases: TypedDict
The serializer context handed to :meth:BatchCollector.collect.
In practice this is view.get_serializer_context() (request,
view and format keys), plus collected: the maps returned by
collect are injected back into it before the rows are validated.
Retry policies¶
RetryPolicy ¶
Base class: run the write function once (the identity policy).
run receives a zero-argument callable (the atomic write) and must
return its result; any exception it does not handle propagates.
FixedRetryPolicy ¶
Bases: RetryPolicy
Retry the write up to attempts times on retry_on exceptions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
attempts
|
int
|
total number of attempts ( |
3
|
delay
|
float
|
seconds to sleep between attempts ( |
0.1
|
retry_on
|
tuple[type[BaseException], ...]
|
exception types that trigger a retry; any other exception propagates immediately. |
(DatabaseError,)
|
never_retry
|
tuple[type[BaseException], ...]
|
exception types that propagate even when they match
Inside an outer transaction ( A caveat: a |
(IntegrityError,)
|
Exceptions¶
BatchSizeLimitExceeded ¶
Bases: APIException
The batch payload exceeds batch_max_size (HTTP 413).
BatchCollectionTimeout ¶
Bases: APIException
The async collector did not finish within COLLECT_TIMEOUT (HTTP 504).
BatchNotFound ¶
Bases: APIException
An update/destroy target pk was not found in the database (HTTP 404).
The detail is the index-aligned row list ({} for rows that would
have applied), matching the batch error format.
BatchPermissionDenied ¶
Bases: APIException
An update/destroy target fails the view's object permissions (HTTP 403).
The detail is the index-aligned row list ({} for rows that would
have applied), matching the batch error format.
BatchConflict ¶
Bases: APIException
A row conflicts with another row in the batch or with existing data (HTTP 409).
The detail is the index-aligned row list ({} for rows that would
have applied), matching the batch error format.
Event loop (drf_batch.loop)¶
Event-loop bridge for collection.
Django (and therefore DRF) dispatches viewsets synchronously, under WSGI
and ASGI alike, so the collection pass needs a way to run its coroutine
from a synchronous thread. run_in_loop(coro) submits it to a loop
running in another thread and blocks until it resolves. The loop is
chosen per call (EVENT_LOOP setting):
"auto"(default): under ASGI, the server's own loop (the sync view runs in anasgirefworker thread, which knows the loop that spawned it), so collection shares the loop the app's async clients live on. Anywhere else, the thread-mode loop below.-
"thread": always the thread-mode loop: -
a loop the application already runs in its own thread, registered with :func:
use_loopor theTHREAD_LOOPsetting (a dotted path to a zero-argument callable returning it); - otherwise the built-in
drf-batch-loopdaemon thread: one event loop for the process, created lazily on first use (importing this module starts no thread), reference-counted persession(), kept running until :func:shutdown(also at exit and recreated afterfork()).
A loop drf-batch does not own (ASGI or user) is never stopped by
:func:shutdown. Every loop must be running in a thread other than the
caller's.
Django ORM connections are thread-local. :func:to_thread runs a blocking
call (notably an ORM query) on a worker thread and applies Django's
request-boundary connection handling around it
(close_old_connections(): CONN_MAX_AGE and health checks apply), so
worker threads reuse connections only as long as the settings allow.
use_loop ¶
Run thread-mode collection on loop, already running in your thread.
Use it when the application owns a loop (and the async clients created
on it) in a thread of its own. It takes precedence over the
THREAD_LOOP setting; None restores the built-in
drf-batch-loop thread. drf-batch never starts, stops or closes
loop. Under ASGI with EVENT_LOOP = "auto" the server's loop is
still preferred.
current_loop ¶
The loop :func:run_in_loop would use from the calling thread.
Lets the application create loop-bound clients on it (with
asyncio.run_coroutine_threadsafe). Starts the built-in thread if
that is the loop in use.
run_sync
async
¶
Run a blocking call where it can see the request's database state.
Inside collection started with serve_caller=True (the request's
connection is in a transaction) the call runs on the request thread,
which serves such calls one at a time while it waits, so it sees the
transaction's uncommitted writes. Anywhere else it runs on a worker
thread, like :func:to_thread.
to_thread
async
¶
Run a blocking call (e.g. a Django ORM query) on the thread pool.
Like :func:asyncio.to_thread, with Django's connection handling around
the call (close_old_connections()): with CONN_MAX_AGE = 0 (the
default) the worker's connection is closed afterwards; with a positive
CONN_MAX_AGE it is reused until it expires or breaks. The call runs
on its own connection: it does not see uncommitted writes of the
request's transaction.
run_in_loop ¶
Run coro on the selected loop and block until it resolves.
coro is a coroutine (an async def collect result, the collection
graph, or any awaitable a sync collect returns). Raises
RuntimeError if the calling thread already has a running event
loop: call the synchronous code through asgiref's
sync_to_async there (which then runs collection on that loop), or
await :func:wait directly. Raises
:class:~drf_batch.exceptions.BatchCollectionTimeout if the result is
not available within timeout seconds; any other exception raised
by coro propagates unchanged.
With serve_caller=True the calling thread does not just wait: it
runs the blocking calls coro hands it through :func:run_sync, one
at a time, so they use this thread's database connection (and see its
open transaction).
wait
async
¶
Await coro directly, for callers already inside a running loop.
Maps asyncio's timeout to
:class:~drf_batch.exceptions.BatchCollectionTimeout.
session ¶
Context manager for one use of the built-in, refcounted loop.
shutdown ¶
Stop the built-in loop thread and release its state (also runs at exit).