Databento¶

Load Databento MBO records into the canonical per-order events frame.

Satisfies the :class:~ob_analytics.protocols.EventLoader protocol.

The fills the loader pairs with their book events are kept on :attr:trade_records for :class:DatabentoTradeReader to project into the trades frame — the same arrangement :class:~ob_analytics.lobster.LobsterSource uses for the companion orderbook file, and for the same reason: the second table is a by-product of reading the first, and reading the file twice to get it would be wasted work on a multi-million-record window.

Parameters:

load(source: Any) -> pd.DataFrame

Build trades from the fill and trade records of a Databento MBO file.

Satisfies the :class:~ob_analytics.protocols.TradeSource protocol.

Which record makes a trade is :attr:DatabentoSettings.trades_from:

• Fills (F), the default. A fill names the resting order, so each one becomes a trade row with a maker and a maker_event_id, one row per resting order a sweep took out — the same per-maker-leg shape the LOBSTER and Bitstamp readers produce. A file with no fills falls back to its prints. The catch: a trade the publisher sent no fill for — an auction, a trade against a non-displayed order, an off-exchange print — has nothing to become, so it is not in the frame. The reader warns with the volume left out.
• Trade prints (T). The venue's whole tape, one row per print, but no maker on any row.

The two are not mixed. A print and the fills behind it describe one execution, and nothing in a DBN record ties them together reliably — a fill and the modify it causes can carry different receive times — so joining them would risk counting the same volume twice.

Where the venue states the aggressor, direction is its answer rather than an estimate. Where it does not, the pipeline classifies it against the reconstructed quotes.

The taker's own order is not identified. A DBN trade record does not reliably carry the aggressing order's id, so taker and taker_event_id are NA. That is worth knowing before reading :func:~ob_analytics.analytics.set_order_types: with no taker ids it labels executed resting orders resting-limit and never market or market-limit.

Parameters:

load(events: DataFrame, source: Any) -> pd.DataFrame

Write an events frame back out as a DBN file of MBO records.

Satisfies the :class:~ob_analytics.protocols.DataWriter protocol, and inverts :class:DatabentoLoader record for record: a created row becomes A, a changed row M, a deleted row C removing the outstanding size, and a non-zero fill becomes an F record immediately before the row that carries it.

The point is a round trip — reading a window, working on it, and writing something another DBN reader can open — not re-creating a vendor file byte for byte. The metadata says OB.ANALYTICS (or whatever :attr:DatabentoSettings.dataset says) rather than claiming to be Databento's own data.

Parameters:

write(
    data: dict[str, DataFrame],
    dest: str | Path,
    *,
    symbol: str = "SYMBOL",
    **kwargs: Any,
) -> Path

The Databento source: offline replay of DBN market-by-order files (L3).

Offline only — Databento's Live client is not wired up here — so it satisfies :class:~ob_analytics.protocols.OfflineSource and has no live capability. Use it as::

from ob_analytics import Pipeline
from ob_analytics.databento import DatabentoSource, DatabentoSettings

source = DatabentoSource(settings=DatabentoSettings(raw_symbol="AAPL"))
result = Pipeline(source=source).run("xnas-itch-20240403.mbo.dbn.zst")

The defaults suit a US equity: a one-cent tick and whole shares. Another instrument needs its own, e.g. PipelineConfig(tick_size=0.25, price_decimals=2) for the E-mini S&P 500 future.

dbn_settings() -> DatabentoSettings

Bases: SourceSettings

Typed settings for :class:DatabentoSource.

A DBN file can hold several instruments and, for a consolidated dataset such as DBEQ.BASIC, several publishers of the same instrument. Each (instrument, publisher) pair is a book of its own, so a run covers exactly one of them: name it here, or the loader raises and lists what the file holds. A file with one instrument and one publisher needs no setting.

Attributes: