Skip to content

Protocol Interfaces

Contracts that pluggable pipeline components must satisfy. Uses structural (duck) typing — implement the right method signature and it works, no inheritance required.

Protocol Method Purpose
EventLoader load(source) → DataFrame Parse raw L3 data into events
DepthSource load(source) → DataFrame Load an L2 price-level feed into the depth frame
TradeSource load(events, source) → DataFrame Build the canonical trades DataFrame
DataWriter write(data, dest) Serialize pipeline outputs
Format factory methods Bundle loader, trade source, and writer for a venue

A Format declares two axes: a FeedType (matched_book vs diff_feed; see Data quality) and a Level resolution (L2 vs L3; see Process L2 feeds).

Level

Bases: str, Enum

Order-book resolution a feed (or a plot) works at — MBP vs MBO.

The granularity axis, orthogonal to :class:FeedType's crossing invariant. A format declares its :attr:Format.resolution so the pipeline knows which stages apply:

  • :attr:L2 — Market-By-Price (MBP): aggregate volume per price level, with no persistent order identity. Price-level feeds (Binance, Kalshi, Polymarket, most CCXT sources) are L2. The per-order stages (:func:~ob_analytics.analytics.set_order_types, :func:~ob_analytics.analytics.order_aggressiveness, queue reconstruction) have nothing to key on and are skipped; depth / spread / trade analytics run directly on the price-level book.
  • :attr:L3 — Market-By-Order (MBO): one primitive per resting order, with stable identity (queue position recoverable). The reconstruction model ob-analytics was built for (Bitstamp, LOBSTER, Databento).

The str mixin lets members slot directly into the visualization renderer-registry tuple keys and, via the :meth:__str__ override, render as the bare token ("L2") in file stems and f-strings rather than "Level.L2".

FeedType

Bases: str, Enum

How a data feed represents the order book — its crossing invariant.

A format declares its feed type so downstream code can reason about crossed books by coordinate, not by format name. The distinction is a property of the source, not of the reconstruction:

  • :attr:MATCHED_BOOK — an L3 feed emitted by the venue's own matching engine (LOBSTER, exchange MBO such as Databento). Bids can never rest above asks, so an uncrossed book is a guaranteed invariant of the data.
  • :attr:DIFF_FEED — an L3 feed reconstructed from a public placement/cancellation diff stream (the Bitstamp public feed). It can contain genuinely crossed resting orders (a bid resting above an ask, neither filling); :func:~ob_analytics.analytics.order_book replays this faithfully — a crossed book in the output is a property of the feed, not a reconstruction bug.
  • :attr:UNKNOWN — a format that does not declare its feed type (the structural default for third-party formats predating this attribute).

Mixes in str so members compare and serialise as their value (FeedType.DIFF_FEED == "diff_feed"), which keeps CLI/JSON output and equality checks ergonomic.

EventLoader

Bases: Protocol

Loads raw order-book events from a data source.

The returned DataFrame must contain at least the columns required by ob_analytics.schemas.validate_events_df.

load

load(source: Any) -> pd.DataFrame

Load events from source and return a DataFrame.

Parameters:

Name Type Description Default
source Any

Data source identifier. The canonical type is str | Path (a file path), but loaders may accept richer descriptors such as dicts, dataclasses, or connection strings.

required

Returns:

Type Description
DataFrame

Events with at least the columns required by ob_analytics.schemas.validate_events_df.

DepthSource

Bases: Protocol

Loads a price-level (L2) depth stream into the canonical depth frame.

The L2 counterpart to :class:EventLoader. Where an EventLoader returns per-order events that the pipeline later folds into depth via :func:~ob_analytics.depth.price_level_volume, a DepthSource returns the depth frame directly — a price-level feed is a depth stream, so there is nothing to reconstruct.

An :attr:Format.resolution of :attr:Level.L2 format's :meth:Format.create_loader returns a DepthSource; the pipeline validates its output with :func:~ob_analytics.schemas.validate_depth_df and feeds it straight to :class:~ob_analytics.depth.DepthMetricsEngine.

Returned DataFrame columns (see :data:~ob_analytics.schemas.DEPTH_COLUMNS):

  • timestamp — pandas datetime64[ns]
  • price — float, the price level
  • volume — float, the level's new absolute resting size after the update (0 removes the level); not a signed delta
  • direction — categorical bid/ask

load

load(source: Any) -> pd.DataFrame

Load a price-level depth stream from source and return the frame.

Parameters:

Name Type Description Default
source Any

Data source identifier — canonically a str | Path (a snapshot + price-level-delta file), but implementations may accept richer descriptors.

required

Returns:

Type Description
DataFrame

Depth with at least the columns required by :func:~ob_analytics.schemas.validate_depth_df.

TradeSource

Bases: Protocol

Builds the trades DataFrame for a given run.

Implementations read explicit trade records (a separate trades.csv, LOBSTER execution rows embedded in the events frame, etc.) and project them into the canonical trades schema.

Returned DataFrame columns:

  • timestamp — pandas datetime64[ns]
  • price — float
  • volume — float
  • direction — categorical buy/sell (taker side)
  • maker_event_id — integer event id of the resting order
  • taker_event_id — integer event id of the aggressing order
  • maker — order id of the resting order
  • taker — order id of the aggressing order
  • maker_og — original_number of the maker event
  • taker_og — original_number of the taker event

load

load(events: DataFrame, source: Any) -> pd.DataFrame

Build the trades DataFrame.

Parameters:

Name Type Description Default
events DataFrame

The processed events frame (post-loader).

required
source Any

The same source value passed to :meth:EventLoader.load. Used by file-based readers to locate companion files.

required

Returns:

Type Description
DataFrame

DataWriter

Bases: Protocol

Writes pipeline results to a format-specific output.

write

write(
    data: dict[str, DataFrame],
    dest: str | Path,
    **kwargs: Any,
) -> Path | tuple[Path, ...]

Write pipeline DataFrames to dest.

Parameters:

Name Type Description Default
data dict of str to DataFrame

Pipeline output keyed by name (e.g. "events", "trades", "depth", "depth_summary").

required
dest str or Path

Output path (file or directory, format-dependent).

required

Format

Bases: Protocol

Structural contract for a data-format descriptor.

A Format bundles the per-format factories the pipeline needs: how to load events, how to acquire trades, and (optionally) how to write results or compute depth directly. Pass instances to Pipeline(format=...).

There is no base class to inherit — any object providing these members satisfies the contract (structural typing). name is a short lowercase identifier (e.g. "bitstamp"). feed_type declares the source's crossing invariant (:class:FeedType); callers should treat a missing attribute as :attr:FeedType.UNKNOWN (structural default) rather than special-casing format names. resolution declares the granularity (:class:Level): :attr:Level.L3 (default) for per-order feeds, :attr:Level.L2 for price-level feeds — callers should treat a missing attribute as :attr:Level.L3 (structural default).

create_loader

create_loader(
    config: Any, ctx: RunContext
) -> EventLoader | DepthSource

Return the loader for this format.

An :attr:Level.L3 format returns an :class:EventLoader (per-order events); an :attr:Level.L2 format returns a :class:DepthSource (the price-level depth frame directly).

create_trade_source

create_trade_source(
    config: Any, ctx: RunContext
) -> TradeSource

Return a trade source for this format.

create_writer

create_writer(
    config: Any, ctx: RunContext
) -> DataWriter | None

Return a writer for this format, or None if unsupported.

compute_depth

compute_depth(
    events: DataFrame,
    config: Any,
    source: Any,
    ctx: RunContext,
) -> tuple[pd.DataFrame, pd.DataFrame] | None

Return (depth, depth_summary) to override the standard depth pipeline, or None to use it.

config_defaults

config_defaults() -> dict[str, Any]

Return default :class:PipelineConfig overrides for this format.

required_context

required_context() -> list[str]

:class:RunContext field names this format requires.

E.g. LOBSTER returns ["trading_date"] because its filenames carry no date; Bitstamp returns []. Lets the CLI/pipeline validate required context generically instead of special-casing format names. Callers should treat a missing method as [] (structural default).