Skip to content

LOBSTER

Support for LOBSTER message files (event types 1–7), orderbook-backed depth, and round-trip writers.

Use via Pipeline(source=LobsterSource(...)) or Pipeline.from_source("lobster", ...).

Key differences from Bitstamp:

  • Executions in the message file — LobsterTradeReader builds trades directly from type 4/5 rows in the events DataFrame (no separate trades file).
  • Orderbook-backed depth — LobsterSource.compute_depth reads the official orderbook file for ground-truth depth instead of reconstructing from events
  • Integer prices — raw prices are in ten-thousandths of a dollar (price_divisor=10000)

LobsterLoader 

LobsterLoader(
    config: PipelineConfig | None = None,
    *,
    trading_date: str | Timestamp,
    session_tz: str = LOBSTER_DEFAULT_TZ,
    venue: str | None = None,
    symbol: str | None = None,
)

Load raw limit-order events from LOBSTER message files.

Satisfies the :class:~ob_analytics.protocols.EventLoader protocol.

Parameters:

Name Type Description Default
config PipelineConfig

Pipeline configuration.

 None
trading_date str or Timestamp

The calendar date of the trading session (LOBSTER timestamps are seconds after midnight and need a date anchor).

 required
session_tz str

The venue's local time zone, used to place the session's seconds after midnight on the shared UTC clock. Defaults to :data:LOBSTER_DEFAULT_TZ ("America/New_York"), correct for LOBSTER's US equity data.

 LOBSTER_DEFAULT_TZ
venue str

Optional instrument identity. When either is supplied, the loaded frame gains per-row venue / symbol columns; venue falls back to "lobster" when only symbol is given. Both None (the default) leaves the frame untagged. The instrument ticker lives in the LOBSTER filename, so pass it as symbol when you want it on the rows.

 None
symbol str

Optional instrument identity. When either is supplied, the loaded frame gains per-row venue / symbol columns; venue falls back to "lobster" when only symbol is given. Both None (the default) leaves the frame untagged. The instrument ticker lives in the LOBSTER filename, so pass it as symbol when you want it on the rows.

 None

load 

load(source: Any) -> pd.DataFrame

Load LOBSTER message data and return a cleaned events DataFrame.

Parameters:

Name Type Description Default
source str, Path, or directory

Path to a LOBSTER message CSV, or a directory containing message/orderbook file pairs. When a directory is given the loader auto-discovers files by the LOBSTER naming convention.

 required

Returns:

Type Description
DataFrame

LobsterTradeReader 

LobsterTradeReader(config: PipelineConfig | None = None)

Build trades directly from LOBSTER execution events.

In LOBSTER, each execution event (type 4 or 5) represents the resting (maker) side of a trade. This reader builds trade records directly from those rows in the events frame; no matching is needed because the data already pairs maker rows with executions.

Satisfies the :class:~ob_analytics.protocols.TradeSource protocol.

load 

load(events: DataFrame, source: Any) -> pd.DataFrame

Build a trades DataFrame from LOBSTER execution events.

Parameters:

Name Type Description Default
events DataFrame

Events with raw_event_type column populated.

 required
source Any

Unused; trade information is embedded in events.

 required

Returns:

Type Description
DataFrame

Trades with timestamp, price, volume, direction, maker_event_id, taker_event_id, maker, taker.

LobsterWriter 

LobsterWriter(
    config: PipelineConfig | None = None,
    *,
    trading_date: str | Timestamp,
    session_tz: str = LOBSTER_DEFAULT_TZ,
    price_divisor: int | None = None,
)

Write pipeline events back to LOBSTER dual-file format.

Satisfies the :class:~ob_analytics.protocols.DataWriter protocol.

Parameters:

Name Type Description Default
trading_date str or Timestamp

Calendar date of the session.

 required
session_tz str

The venue's local time zone, used to convert the shared UTC clock back to LOBSTER's seconds after local midnight. Defaults to :data:LOBSTER_DEFAULT_TZ; must match the value used on load.

 LOBSTER_DEFAULT_TZ
price_divisor int

Multiplier to convert decimal prices back to LOBSTER integers.

 None

write 

write(
    data: dict[str, DataFrame],
    dest: str | Path,
    *,
    ticker: str = "DATA",
    num_levels: int = 10,
    **kwargs: Any,
) -> tuple[Path, Path]

Write events to LOBSTER message + orderbook files.

Parameters:

Name Type Description Default
data dict

Must contain "events" key.

 required
dest str or Path

Output directory.

 required
ticker str

Ticker symbol for filename.

 'DATA'
num_levels int

Number of orderbook levels to write.

 10

Returns:

Type Description
tuple of Path

(message_path, orderbook_path)

LobsterSource dataclass 

LobsterSource(settings: SourceSettings = SourceSettings())

The LOBSTER source: offline replay of message/orderbook files (L3).

trading_date is taken from the per-run :class:~ob_analytics.protocols.RunContext, not the source constructor — so the same LobsterSource() instance can be reused across runs with different sessions. Offline only: LOBSTER ships as data files, so it satisfies :class:~ob_analytics.protocols.OfflineSource and has no live capability.

lobster_depth_from_orderbook 

lobster_depth_from_orderbook(
    events: DataFrame,
    orderbook_path: Path,
    config: PipelineConfig,
) -> tuple[pd.DataFrame, pd.DataFrame]

Compute depth and depth summary from the LOBSTER orderbook file.

The LOBSTER orderbook file is ground truth: it records the complete visible book state after every message event. This function converts it into the (depth, depth_summary) pair the pipeline expects, avoiding the need to reconstruct depth from message events (which fails when events reference pre-market orders absent from the message file).

Parameters:

Name Type Description Default
events DataFrame

Events DataFrame (used only for timestamps and event IDs).

 required
orderbook_path Path

Path to the LOBSTER orderbook CSV.

 required
config PipelineConfig

Pipeline configuration.

 required

Returns:

Type Description
tuple of (depth DataFrame, depth_summary DataFrame)