Data I/O¶

Parquet serialization and writer registry.

load_data ¶

load_data(path: str | Path) -> dict[str, pd.DataFrame]

Load pre-processed pipeline data from a Parquet folder.

Parameters:

Name	Type	Description	Default
`path`	`str or Path`	A folder written by :func:`save_data` in its default Parquet format. Each `.parquet` file inside is loaded as a DataFrame keyed by its stem (`events.parquet` → `"events"`).	required

Returns:

Type	Description
`dict of str to pandas.DataFrame`

Raises:

Type	Description
`ConfigError`	If path is not a folder, a file in it is not a Parquet file this build can read, or a Parquet file declares a schema version this build does not support. A file written with no version key (legacy data, or the bundled sample) loads with a warning — see :func:`ob_analytics.schemas.check_schema_version`.
`FileNotFoundError`	If the folder holds no Parquet file.

save_data ¶

save_data(
    lob_data: PipelineResult | dict[str, DataFrame],
    path: str | Path,
    *,
    fmt: str = "parquet",
    writer: DataWriter | None = None,
    config: Any = None,
    ctx: Any = None,
    **write_kwargs: Any,
) -> None

Save pipeline data to disk.

Parameters:

Name	Type	Description	Default
`lob_data`	`PipelineResult or dict of str to pandas.DataFrame`	A pipeline result, or the DataFrames to save (keys become file stems). A result is saved as its four tables, with its own config unless config is given, and with its declarations. :func:`load_result` reads a Parquet folder back as the same result.	required
`path`	`str or Path`	A folder for this package's own formats (`"parquet"`, `"depth_csv"`, `"bitstamp"`, `"lobster"`), and a file for an export to another tool (`"hftbacktest"`, `"nautilus"`, `"databento"`). Each writer declares which it takes as its :attr:`~ob_analytics.protocols.DataWriter.target`. A folder is created when missing.	required
`fmt`	`str`	The format, one of :func:`list_writers`. The default, `"parquet"`, writes one file per key and tags each with :data:`SCHEMA_VERSION` in its metadata (checked by :func:`load_data`). A source name (e.g. `"bitstamp"`, `"lobster"`) round-trips through that source's own writer (its `create_writer` method, so no separate writer registration is needed); a source that needs run state — LOBSTER's `trading_date` — reads it from ctx. A generic, source-independent writer registered via :func:`register_writer` is also resolved by name, and wins over a source of the same name.	`'parquet'`
`writer`	`DataWriter`	A pre-constructed writer instance. When provided, fmt is ignored and the writer is used directly.	`None`
`config`	`Any`	Forwarded to a registered writer factory when `fmt` names one. The `"parquet"` writer records config in each file, so the files keep the run's tick size, lot size and display precision. `ctx` defaults to an empty :class:`~ob_analytics.protocols.RunContext`.	`None`
`ctx`	`Any`	Forwarded to a registered writer factory when `fmt` names one. The `"parquet"` writer records config in each file, so the files keep the run's tick size, lot size and display precision. `ctx` defaults to an empty :class:`~ob_analytics.protocols.RunContext`.	`None`
`**write_kwargs`	`Any`	Extra keyword arguments forwarded to `writer.write()`.	`{}`

Raises:

Type	Description
`UnknownNameError`	If fmt is not one of :func:`list_writers`.
`ConfigError`	If path is a file where the writer takes a folder, or a folder where it takes a file. The message says which the writer expected. Also if the writer declares no `target`.

load_result ¶

load_result(
    path: str | Path, *, source: Any = None
) -> PipelineResult

Load a saved run back into a :class:~ob_analytics.pipeline.PipelineResult.

The inverse of save_data(result, path): the four tables come back as they were saved, and so does the run's :class:~ob_analytics.config.PipelineConfig, which each Parquet file records. A result rebuilt this way shows the same prices and sizes as the run did, for example in :func:~ob_analytics.visualization.display_result or a gallery.

A folder that records only the tick and lot size (written by :meth:~ob_analytics.pipeline.Pipeline.run_windows, or by an older version) gets those two, with the display precision that shows one step of each. A folder with neither (prices and sizes stored as floats) gets a tick size of 1, so its prices read as they are.

The resolution comes from the tables: an :attr:~ob_analytics.protocols.Level.L2 run has an empty events table, and an L3 run never does.

The run's declarations (:attr:PipelineResult.declarations) come back too. A folder written by an older version does not record them; it gets those of the capture record ob-analytics process keeps beside the tables, over those of source (:func:~ob_analytics.capture_record.run_declarations).

Parameters:

Name	Type	Description	Default
`path`	`str or Path`	A Parquet folder written by :func:`save_data` or by `ob-analytics process`.	required
`source`	`Source`	The source whose declarations to use when the folder records none. `None` leaves them undeclared. Recorded declarations always win, since they describe the data the run read.	`None`

Returns:

Type	Description
`PipelineResult`	The saved run.

Raises:

Type	Description
`ConfigError`	If path is not a folder (it does not exist, or it is a file), if the folder lacks one of the four tables, or if a Parquet file declares a schema version this build does not support.
`FileNotFoundError`	If the folder holds no Parquet file at all.

register_writer ¶

register_writer(name: str, factory: WriterFactory) -> None

Register a writer factory under name for use with save_data(fmt=name, ctx=...).

The factory is called as factory(config, ctx) and must return a :class:DataWriter. This is what lets format-specific writers (e.g. :class:~ob_analytics.lobster.LobsterWriter, which needs trading_date) participate in the registry — they pull required parameters from the :class:~ob_analytics.protocols.RunContext.

list_writers ¶

list_writers() -> list[str]

Return every format :func:save_data accepts, sorted.

The formats are the writers registered via :func:register_writer and the sources that write their own format (those with a create_writer method). This is the list save_data checks fmt against and names in its error.

load_writer_plugins ¶

load_writer_plugins(*, force: bool = False) -> list[str]

Discover and register writers advertised via entry points.

Scans the :data:ENTRY_POINT_GROUP entry-point group; each entry's value is loaded to a writer factory (config, ctx) -> DataWriter and registered under the entry-point name, as :func:register_writer would. This is what lets a writer live in a separate installable package without editing ob-analytics.

The scan runs once per process unless force is set. A plug-in that fails to load is skipped with a warning, so one broken package cannot stop the rest from loading.

Returns:

Type	Description
`list of str`	The names newly registered by this call.