Protocol Interfaces¶
Contracts that pluggable pipeline components must satisfy. Uses structural (duck) typing — implement the right method signature and it works, no inheritance required.
| Protocol | Method | Purpose |
|---|---|---|
EventLoader |
load(source) → DataFrame |
Parse raw L3 data into events |
DepthSource |
load(source) → DataFrame |
Load an L2 price-level feed into the depth frame |
TradeSource |
load(events, source) → DataFrame |
Build the canonical trades DataFrame |
DataWriter |
write(data, dest) |
Serialize pipeline outputs |
Format |
factory methods | Bundle loader, trade source, and writer for a venue |
A Format declares two axes: a FeedType (matched_book vs diff_feed; see
Data quality) and a Level resolution (L2 vs L3; see
Process L2 feeds).
Level ¶
Bases: str, Enum
Order-book resolution a feed (or a plot) works at — MBP vs MBO.
The granularity axis, orthogonal to :class:FeedType's crossing
invariant. A format declares its :attr:Format.resolution so the
pipeline knows which stages apply:
- :attr:
L2— Market-By-Price (MBP): aggregate volume per price level, with no persistent order identity. Price-level feeds (Binance, Kalshi, Polymarket, most CCXT sources) are L2. The per-order stages (:func:~ob_analytics.analytics.set_order_types, :func:~ob_analytics.analytics.order_aggressiveness, queue reconstruction) have nothing to key on and are skipped; depth / spread / trade analytics run directly on the price-level book. - :attr:
L3— Market-By-Order (MBO): one primitive per resting order, with stable identity (queue position recoverable). The reconstruction model ob-analytics was built for (Bitstamp, LOBSTER, Databento).
The str mixin lets members slot directly into the visualization
renderer-registry tuple keys and, via the :meth:__str__ override, render
as the bare token ("L2") in file stems and f-strings rather than
"Level.L2".
FeedType ¶
Bases: str, Enum
How a data feed represents the order book — its crossing invariant.
A format declares its feed type so downstream code can reason about crossed books by coordinate, not by format name. The distinction is a property of the source, not of the reconstruction:
- :attr:
MATCHED_BOOK— an L3 feed emitted by the venue's own matching engine (LOBSTER, exchange MBO such as Databento). Bids can never rest above asks, so an uncrossed book is a guaranteed invariant of the data. - :attr:
DIFF_FEED— an L3 feed reconstructed from a public placement/cancellation diff stream (the Bitstamp public feed). It can contain genuinely crossed resting orders (a bid resting above an ask, neither filling); :func:~ob_analytics.analytics.order_bookreplays this faithfully — a crossed book in the output is a property of the feed, not a reconstruction bug. - :attr:
UNKNOWN— a format that does not declare its feed type (the structural default for third-party formats predating this attribute).
Mixes in str so members compare and serialise as their value
(FeedType.DIFF_FEED == "diff_feed"), which keeps CLI/JSON output and
equality checks ergonomic.
EventLoader ¶
Bases: Protocol
Loads raw order-book events from a data source.
The returned DataFrame must contain at least the columns required by
ob_analytics.schemas.validate_events_df.
load ¶
Load events from source and return a DataFrame.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source
|
Any
|
Data source identifier. The canonical type is |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
Events with at least the columns required by
|
DepthSource ¶
Bases: Protocol
Loads a price-level (L2) depth stream into the canonical depth frame.
The L2 counterpart to :class:EventLoader. Where an EventLoader
returns per-order events that the pipeline later folds into depth via
:func:~ob_analytics.depth.price_level_volume, a DepthSource returns
the depth frame directly — a price-level feed is a depth stream, so
there is nothing to reconstruct.
An :attr:Format.resolution of :attr:Level.L2 format's
:meth:Format.create_loader returns a DepthSource; the pipeline
validates its output with
:func:~ob_analytics.schemas.validate_depth_df and feeds it straight to
:class:~ob_analytics.depth.DepthMetricsEngine.
Returned DataFrame columns (see
:data:~ob_analytics.schemas.DEPTH_COLUMNS):
timestamp— pandas datetime64[ns]price— float, the price levelvolume— float, the level's new absolute resting size after the update (0removes the level); not a signed deltadirection— categoricalbid/ask
load ¶
Load a price-level depth stream from source and return the frame.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source
|
Any
|
Data source identifier — canonically a |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
Depth with at least the columns required by
:func: |
TradeSource ¶
Bases: Protocol
Builds the trades DataFrame for a given run.
Implementations read explicit trade records (a separate
trades.csv, LOBSTER execution rows embedded in the events
frame, etc.) and project them into the canonical trades schema.
Returned DataFrame columns:
timestamp— pandas datetime64[ns]price— floatvolume— floatdirection— categoricalbuy/sell(taker side)maker_event_id— integer event id of the resting ordertaker_event_id— integer event id of the aggressing ordermaker— order id of the resting ordertaker— order id of the aggressing ordermaker_og— original_number of the maker eventtaker_og— original_number of the taker event
load ¶
Build the trades DataFrame.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
events
|
DataFrame
|
The processed events frame (post-loader). |
required |
source
|
Any
|
The same |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
|
DataWriter ¶
Bases: Protocol
Writes pipeline results to a format-specific output.
write ¶
Write pipeline DataFrames to dest.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
dict of str to DataFrame
|
Pipeline output keyed by name (e.g. |
required |
dest
|
str or Path
|
Output path (file or directory, format-dependent). |
required |
Format ¶
Bases: Protocol
Structural contract for a data-format descriptor.
A Format bundles the per-format factories the pipeline needs: how to
load events, how to acquire trades, and (optionally) how to write
results or compute depth directly. Pass instances to
Pipeline(format=...).
There is no base class to inherit — any object providing these
members satisfies the contract (structural typing). name is a
short lowercase identifier (e.g. "bitstamp"). feed_type
declares the source's crossing invariant (:class:FeedType); callers
should treat a missing attribute as :attr:FeedType.UNKNOWN (structural
default) rather than special-casing format names. resolution
declares the granularity (:class:Level): :attr:Level.L3 (default) for
per-order feeds, :attr:Level.L2 for price-level feeds — callers should
treat a missing attribute as :attr:Level.L3 (structural default).
create_loader ¶
Return the loader for this format.
An :attr:Level.L3 format returns an :class:EventLoader (per-order
events); an :attr:Level.L2 format returns a :class:DepthSource
(the price-level depth frame directly).
create_trade_source ¶
Return a trade source for this format.
create_writer ¶
Return a writer for this format, or None if unsupported.
compute_depth ¶
compute_depth(
events: DataFrame,
config: Any,
source: Any,
ctx: RunContext,
) -> tuple[pd.DataFrame, pd.DataFrame] | None
Return (depth, depth_summary) to override the standard
depth pipeline, or None to use it.
config_defaults ¶
Return default :class:PipelineConfig overrides for this format.
required_context ¶
:class:RunContext field names this format requires.
E.g. LOBSTER returns ["trading_date"] because its filenames carry
no date; Bitstamp returns []. Lets the CLI/pipeline validate
required context generically instead of special-casing format names.
Callers should treat a missing method as [] (structural default).