Changelog¶

All notable changes to ob-analytics are documented in this file.

The format is based on Keep a Changelog.

[Unreleased]¶

Added¶

load_result(path) reads a saved run back as a PipelineResult (#346). It is the inverse of save_data(result, path): the four tables come back, and so does the run's PipelineConfig. save_data now takes the result itself, and each Parquet file records the run's whole config under the new key ob_analytics_config. A folder that records only the tick and lot size (a windowed run, or one saved by an older version) gets those two, with the display precision that shows one step of each; a folder of whole-number prices or sizes that records neither is warned about. load_result reads Parquet folders only, and refuses a pickle, which keeps no config. The key is optional, so the schema version stays 4.0 and older readers ignore it.
ob_analytics.capture_record owns a capture's meta.json (#346). read_record(path) returns a typed CaptureRecord: the source that made the capture, its feed type, trade attribution, sequence kind and clocks, the sequence restarts, and the instrument's tick size and lot size. read_instrument(path) returns only the tick and lot size, as config fields. write_record and source_declarations are what the capture runner and the segment supervisor write it with. The module is outside ob_analytics.live, so code that only replays files does not import the live package.
A ccxt capture records the venue's size step as lot_size (#346), from ccxt's market data, beside the tick size. If a size arrives between two steps, the capture makes the step finer until the size fits, as it does for the tick, and counts the changes in lot_size_changes. The recorded lot is never finer than 1e-8, the default grid.
audit counts trades matched by order id (#352). A new summary field, trades_matched_by_order, counts the trades whose taker is an order created with size 0. Bitstamp sends an instant order bought by quote amount that way and never reports what it bought, so its trades are matched to it by its order id, not by size. audit prints the count as matched by order and reports it as information; it never fails a run. A feed that does not show takers prints not checked. The bundled Bitstamp sample has 2. schemas.unsized_order_ids(events) returns the ids of orders created with size 0; the trade reader and audit both use it.
A source can take an API key (#239). A source declares each key it needs as a SecretStr | None field of its settings, marked with Credential(env=..., issued_at=...). The settings read an unset key from that environment variable; with from_file=True the variable holds the path of a key file. A key is never a command-line option. run_capture stops before it writes anything when a key is missing, and names the variable and where the venue issues keys; a key shorter than eight characters is refused. No file a capture writes holds a key, and neither does the CLI log: a key that reaches one, in a venue's message or an error, is replaced with **********. ob-analytics sources lists the variables each source needs. No built-in source needs a key yet. See "Use an API key".
Exercises and an editable toy book in the tutorial (#121). Chapters 2, 4 and 6 end with exercises: where a new order queues and which order keeps the Bitstamp book crossed (chapter 2), what happens if Eve does not cancel (chapter 4), and how the VPIN bucket size changes the result (chapter 6). Each solution is collapsed until the reader opens it, and runs when the documentation is built, so a wrong answer fails the build. The toy session page now has a book replay to step through, and shows how to rewrite the session: toy_orders() returns its twelve orders as ToyOrders keyed by actor, and match_toy_orders(orders) matches an edited script by price–time priority and returns its events and trades in the layout of toy_events() and toy_trades().
Run a large input one time window at a time (#116). Pipeline.run_windows(source, boundaries, output) cuts one input at the given times and runs the depth stages on one window at a time, so their peak memory is set by the largest window. Each window is written to output as it finishes, as one Parquet file per table, the same folder save_data writes; load_data reads it back. With carry=True, the default, each window starts from the book the previous one ended with, and the output matches a single run row for row. The exceptions are aggressiveness_bps on a Bitstamp input, and depth where the Databento loader already warns that it is off. A run that fails part-way leaves the output folder as it was. On 1.26M events, eight windows peak at 771 MiB against 1,507 MiB for a single run. The loader still reads the whole input. See "Scale and chunking".
The flow-toxicity faces and the L1 quote are built-in metrics (#118). vpin, kyle_lambda, order_flow_imbalance and ofi_horizon are now registered metrics, so available_concepts lists them and result.plot("vpin") computes and draws VPIN with the defaults of compute_vpin. A keyword goes to the calculation or to the picture, whichever names it: result.plot("vpin", n_buckets=20, threshold=0.8). result.metric(name, **settings) takes the calculation's settings too. The new l1_ticker metric is the Level 1 quote — best bid, best ask and last trade. It draws the three prices over time, or the quote card for one instant with at=; plot("l1_ticker", bid=99, ask=101, last=100) draws a card from plain numbers. Each built-in metric gets a gallery card. ofi_by_horizon is the multi-horizon order flow imbalance that the ofi_horizon face draws, now a function in flow_toxicity.
A book replay (#117). The book_replay concept steps through the book over part of a run, on Plotly: the ladder at one instant per second beside the trades and the mid, with a slider and play buttons at 1×, 5× and 20× real time. Click a trade to move to it: the price level it took is marked in the trade's colour. The L2 face replays the depth table, so its touch equals the depth summary's at every frame; the L3 face shows each order, with stale crossed orders removed so both faces show the same market. The gallery shows it for the zoom window, and prepare.book_replay builds it directly. The price-level book the depth summary is built on is now its own class, ob_analytics.depth.PriceLevelBook, and price_level_snapshots returns that book at many instants in one pass.
A page on what each feed shows (#288). "What each feed shows" describes 16 properties that decide what a capture can tell you, such as depth shown, what can be missed, sequence, clocks and taker side. It gives each property's value for every source and venue, checked with live captures. Every source's how-to page now opens with its values and when to use it.
FeedType.PRICE_LEVELS, the crossing value for L2 feeds (#288). The ccxt source, cryptofeed at L2 and depth_csv declare it instead of MATCHED_BOOK. The venue does not publish a crossed price-level book, so audit scores a crossed one as an error, as for a matched book: most often, the capture kept a level the venue removed. Captures made before this record matched_book and are scored the same way. In the Source protocol, level and feed_type are now read-only, so a source can work either one out, as the cryptofeed source does from its venue.
A live capture can run for days (#150). ob-analytics capture now writes a directory of segments with a manifest.json. A lost connection ends the segment; the capture waits (1 s, doubling to 60 s) and starts a new one from a fresh snapshot. --roll-minutes and --roll-mb start a new segment by time or size, and the new segment is streaming before the old one stops, so a roll loses nothing. Running the same command again continues the capture, and closes a segment a crashed process left open; a lock on the directory stops a second process from writing to it at once. The manifest records every segment, why it ended, and every gap with its cause. process and audit read the whole capture, and audit adds the checks capture_gaps, unfinished_segments and dropped_messages. From Python: ob_analytics.live.run_capture and read_manifest.
Each source declares which orders of a trade it can name (#284). The new Source.trade_attribution (TradeAttribution.BOTH, MAKER_ONLY or NONE) says whether a feed's order events show the trade's maker, its taker, or neither. bitstamp names both. lobster, databento and cryptofeed at L3 name the maker only: their feeds show resting orders, and a taker trades on arrival. The L2 sources name neither. The unmatched_trades check now counts only the orders the feed can name, and says which. Before, a Databento run could only report every trade as unmatched, and LOBSTER's guessed takers counted as matches. A source that does not declare it is read as BOTH, the old check.
A capture records what its source declares, and audit holds it to that (#284). meta.json now records source, feed_type and trade_attribution. ob-analytics process copies meta.json into its output. audit checks a capture against the declarations of the source that made it, even when --source names another: a cryptofeed L3 capture is read with --source bitstamp, whose feed shows more. Read the record with recorded_source, recorded_feed_type and recorded_trade_attribution.
A cryptofeed reconnect is counted (#309). cryptofeed reconnects by itself when it loses a message and starts again from a new opening book. The capture counts each new opening book as book_resyncs in meta.json, as the ccxt source already did. On Bitfinex and Blockchain.com the venue's count starts again too; the capture counts that step back as sequence_restarts, and audit leaves it out of sequence_out_of_order, so a reconnect is scored the same on every venue. Read it with recorded_sequence_restarts and pass it to data_quality_summary(sequence_restarts=...).

Changed¶

Pipeline.run reads a capture's tick and lot size (#346). Given a capture folder or a file in one, Pipeline.run and Pipeline.run_windows use the tick_size and lot_size in its meta.json when the config does not set them. So Pipeline.from_source("depth_csv").run(capture) and ob-analytics process capture --source depth_csv give the same result; the first used to raise ConfigError on a capture with a 0.001 tick. A tick size or lot size you set still wins, and a pipeline given its own loader or trade source does not use the record.
The recorded_* readers in ob_analytics.depth_l2 are gone (#346). This can break code that imports them. Use ob_analytics.capture_record.read_record(path) and its fields instead: recorded_tick_size(p) is read_record(p).tick_size, recorded_sequence_kind(p) is read_record(p).sequence_kind, and so on for source, feed_type, trade_attribution, clocks and sequence_restarts. A declaration with a value this version does not know is now left out with a warning, where the old readers raised ValueError.
Bar turnover is a float (#346). It was the integer product of ticks and lots on a pipeline result.
The quote helpers in trade_sign changed (#345). This can break code that called them directly:
prevailing_mid is the mid at an instant, the instant included, from the readable quotes only. Its allow_exact and skip_crossed settings are gone. For the mid a trade arrived into, use the new mid_before(rows, quotes); for the row position of that quote, quote_before(rows, quotes).
readable_quotes moved from ob_analytics.features to ob_analytics.depth.
classify_trade_sign and resolve_direction return the ordered buy/sell categorical of the trades schema.
resolve_direction takes fallback="tick" (the default) or fallback=None, which leaves the unlabelled trades empty when there are no quotes.
A plot drawn on your own axes leaves your figure's layout alone (#120). With ax=, a matplotlib face used to call tight_layout() on the figure that holds the axes. That moved panels you had placed, and warned when the figure held axes tight_layout cannot place, such as a gridspec strip or a colorbar. A face now lays out only a figure it creates. If you draw panels with ax=, call fig.tight_layout() yourself once every panel is drawn. On your axes, the cancellations colorbar now shares your gridspec, so your tight_layout() can place it.
A metric is computed when its plot is drawn (#118). Building the gallery model, available_concepts and result.plot of another concept no longer run every registered metric. A metric that raises now keeps its gallery card, which says why it has no plot, instead of being left out.
A plot concept is level-less or drawn at a level, on every backend (#302). Registering a concept at None when it is registered at a level on any backend, or the reverse, raises a ValueError that says which to use. So does a renderer key that is not (concept, level, backend). RENDERERS.placements(concept) lists how a concept is registered. This can break code that worked before:
Code that registered one plot both ways fails at the second registration.
A metric plug-in that does so is skipped when ob_analytics is imported, with a logged warning.
A backend module loaded with register_plot_backend that registers a built-in concept as the other kind fails to load, so no plot on that backend draws until it is fixed.
A metric named like a built-in plot drawn at a level, such as trade_size, cannot register its level-less renderer; give it another name.
The capture directory layout (#150). A capture's files are now in seg-0001/, seg-0002/, ... under --out, next to manifest.json. Read one segment as before, from its orders.csv or depth.csv; give process and audit the capture directory to read them all. --out must be new, empty, or a capture of the same venue and pair.
Live sources no longer reconnect by themselves (#150). The Bitstamp source reconnected without a new snapshot, so an order deleted while it was disconnected stayed on the book to the end of the run, and only a reconnects count (now removed) recorded it. The ccxt source stopped its book or trade loop on a network error and ran on with half its feed. Both now end the stream with the error, and the capture starts a new segment.
manifest.json counts resyncs, not missing sequence numbers (#309). Each segment, and the capture as a whole, records book_resyncs in place of sequence_missing, and audit warns about them in a new book_resyncs check. dropped now counts only the messages a source could not use. recorded_sequence_kind returns None when a capture records no kind, as recorded_feed_type does, in place of taking a default.

Removed¶

analytics.uncross_book_sides (#342). It uncrossed a book a second way and broke a tie at one price differently from order_book(..., uncross=True), so the book snapshot and the book replay could show different books for one instant. prepare.book_snapshot loses its uncross option with it: it draws the book it is given, so pass it order_book(..., uncross=True) for an uncrossed ladder or depth chart.
order_book(min_bid=, max_ask=) (#342). Neither setting was ever applied. Use bps_range or max_levels, or filter the returned frames.

Fixed¶

Every rebuild puts an order at one price (#342). An order rests at the price of its created row, and after that only a changed row with no fill moves it. The depth table followed this rule, but the per-order book (order_book) and the queue tables put an order at the price of its latest row. Bitstamp reports a taker's fill at the price it traded at, so an order that crossed and then rested showed at one price on the L2 faces and at another on the L3 faces. The rule is now worked out in one place, _engine_frames.resting_price, and order_book, queue_positions, queue_age_grid, price_level_volume, the windowed run's carry and DatabentoWriter all read it. An id that is cancelled and placed again now rests at its new price in every rebuild; the depth table kept it at its first price. A changed row with no fill and a new price (a Databento modify) now moves the order to the back of its new queue in queue_positions and queue_age_grid, as it already moved it in the depth table. queue_positions reports the move as a deleted row at the old price and a created row at the new one. order_book, queue_positions and queue_age_grid now need the fill column, and the two queue functions also need type: classify the events with set_order_types first.
The touch-queue faces leave market orders out (#342). queue_age_grid and queue_positions kept market orders, which never rest. A Bitstamp taker is created at its limit and fills at other prices, so the grid added it and never removed it. On the bundled sample one such order was the liquidity_at_touch L3 bid queue from 02:36 to the end: the grid's touch queue differed from the book at 184 of 200 bid samples and 173 of 200 ask samples. It now matches the book at every sample. On the synthetic golden session, queue_positions loses the 251 rows of its 92 market orders and no other row changes.
A saved run reloads with its own units (#346). ob-analytics gallery and audit --from-parquet rebuilt a saved run with the default lot size and price decimals, so a LOBSTER or Databento run (lot size 1) showed every size 10^8 times too small, and a 0.001 tick was rounded to the cent. They now use load_result.
The L2 loaders warn about a size between two lots (#346). Such a size is rounded to the nearest lot, which is not the venue's size; the warning names lot_size.
A large price or size on its grid is no longer read as off it (#346). The grid test allowed a fixed 1e-6 of a step, but dividing a large value by a small step misses a whole number by more: 1251.81488791 at a lot of 1e-8 is 125181488790.99998 lots. The allowance now grows with the number of steps.
A size too large for int64 lots is refused (#346). Converting a size to lots wrapped a level above about 9.2e10 units (at the default lot of 1e-8) to a negative number, and the L2 loader then dropped the level. It now raises ConfigError that names lot_size.
Bar turnover and VWAP, and trade_impacts' VWAP, no longer wrap (#346). Price in ticks times size in lots passed int64 on a busy day: four trades of 4,000 BTC at 78,000.00 gave a negative turnover and VWAP. They are computed as floats, as amihud already was.
Every Bitstamp trade finds its taker (#352). An instant order bought by quote amount is created with size 0 and then shows 0 or one lot, never what it bought. The trade reader matched a trade to a row only by its fill size, so these trades had no taker and their orders were classed unknown. When no row shows the trade's size and the taker is an order created with size 0, the reader now takes that order's deleted row, else its latest row. Every other trade is still matched on size only: an order with no row of the trade's size has lost a message, and the trade stays unmatched so unmatched_trades reports it. On the bundled sample all 284 takers resolve (282 before), and orders 2002354918830080 and 2002351232385027 are classed market. As market orders they no longer rest in the price-level rebuild, so depth and depth_summary lose their 4 rows (313,565 to 313,561), including the one row whose best bid was a single lot from one of them.
A trade is read against the quote it arrived into, everywhere (#345). Lee–Ready read the quote stamped at the trade's own instant. On a feed whose trades and book events share one clock (LOBSTER, Databento) that quote is the book after the trade took the touch, which pulls the mid toward the trade price and flips signs. With quotes 99/101, then 99/104 at the trade's instant, an unlabelled buy at 101 was labelled sell, and transaction_costs then gave it an effective spread of -2.0. It is now a buy at +2.0. Lee–Ready, transaction_costs, hidden_trades, order_aggressiveness and the windowed run all use one rule, trade_sign.quote_before: the last quote strictly before the row, in event order. On a LOBSTER AAPL hour (30 levels), Lee–Ready now agrees with the feed's own side on 95.9% of trades, against 85.2%; on the bundled Bitstamp sample it is 72.9% against 73.2%. The pipeline's own tables do not change on the bundled Bitstamp sample or the synthetic session, where every trade is labelled by the venue.
hidden_trades reads the spread just before the maker's fill in event order (#345). It read the spread before the fill's whole instant, so a visible order placed or filled earlier in that instant was missed. On the LOBSTER AAPL day (10 levels) it now flags 11,304 of the 11,332 executions against hidden orders, against 9,645, still with none that is not one. On the bundled Bitstamp sample it flags 29 trades, against 41: the 12 it drops each had a visible maker placed at the trade price earlier in the same instant.
One test of a quote that can be read (#345). prevailing_mid and the effective spread kept a quote with an empty side, so a buy at the ask just after the bid side emptied cost 20,000 bps; it is now measured against the last quote with both sides. depth.readable_quotes (moved from features) is the one test: both sides above zero and the bid not above the ask. prevailing_mid, mid_before, hidden_trades, the feature table and the depth heatmap's spread line all use it.
The pipeline fills a missing trade direction by the same rule as the metrics (#345). The pipeline and resolve_direction disagreed on which trades needed a direction (missing, against neither buy nor sell) and on the dtype (ordered against unordered). The pipeline now calls resolve_direction(..., fallback=None), and both write the trades schema's ordered buy/sell categorical. A run with no quotes still leaves the unlabelled trades empty, now with a UserWarning rather than a log line.
LOBSTER depth is stamped with the right event after a cross trade or a halt (#340). The orderbook file has a row for every message, type 6 (cross trade) and type 7 (halt) included, but the events keep only types 1 to 5. lobster_depth_from_orderbook paired the two by position, so after the first type 6 or 7 message every book state carried a later event's time and event_id, and the last states were lost. Each event now reads the orderbook row of its own message, named by original_number. A LobsterSource run raises ConfigError on an orderbook file with more or fewer rows than the message file, where before it logged a warning a Python user never saw. lobster_depth_from_orderbook called on its own checks the count only when given message_rows.
LOBSTER depth sizes are int64 lots (#340). Sizes read from the orderbook file were float64 share counts, so depth.volume and every size column of depth_summary were floats, against the schema, and with a lot_size other than 1 they were in different units from the events. They are now converted with lot_size, as the message sizes are.
LobsterWriter writes the run's own book (#340). The writer rebuilt the orderbook file from the events with rules of its own that only fitted frames read from LOBSTER. On a frame from any other source it raised TypeError (the missing raw_event_type), and without that column it wrote a book that differed from the run's depth. It now writes the book after each event from the depth table it is given, or from price_level_volume when there is none, through PriceLevelBook, the book the depth summary is built on. Reading the files back gives the run's book after every event: the depth summary read back equals the run's, as far as the written levels reach. Both files are written in time order, and an empty level's dummy price is no longer multiplied by the price divisor. The writer refuses, with ConfigError, a run with no events, repeated event_ids, events or depth in display units, and a price_divisor too coarse for the tick (with a cent tick, a divisor of 1 wrote 100.01 and 100.02 as the same price). Events with no depth table now need the type column that set_order_types adds, so events straight from LobsterLoader.load are refused: pass the run's depth, or run set_order_types first. Prices are encoded as exact integers, and a writer built with no config uses LOBSTER's own grid (price_divisor=10_000, whole shares) instead of the general defaults.
One rule for a deleted row, and the loaders and writer that broke it (#341). The schema now says what a deleted row holds when its order also traded: fill is the executed part, volume the part removed without trading, and the two add up to what the order had resting. An order filled in full ends with volume 0. See "Volume and fill" in the schema page.
The Databento loader put the size in both columns, so an order filled in full came off its price level twice and other orders at that price lost their size from depth and depth_summary. Bids of 5 and 3 at 100.00, with the 5 filled, now leave 3 at 100.00, not 0. A cancel that asks to remove more than its order holds now removes only what the order holds.
DatabentoWriter is now the inverse of that rule. A deleted row is a C for everything the order had resting. It writes an F only for a fill the order took while resting. An order that took liquidity when it arrived is written from when it rests, as an A for the size it had left, or not at all if nothing was left. Pass "trades" with the events so the writer knows every fill an aggressor took; without them it knows only the market orders' fills. A synthetic session written to DBN and read back now has the same trades and the same final book; before, every trade came back twice and filled orders stayed resting. A size that is not a whole number of units now raises a ValueError instead of being rounded.
The Bitstamp loader now puts each order's created row first. It sorted an order's rows by size, so an instant order bought by quote amount, created with size 0, had its created row last and a phantom 1-lot fill. A rise in size is no longer read as a fill.
price_level_volume keeps int64 sizes after a partial cancel. One LOBSTER type 2 or Databento partial C made the whole depth.volume column, and the depth_summary sizes, float64.
price_level_volume no longer hides a level the events overdraw. It clipped only the size it reported, so the level stayed short for the rest of the run: overdrawn by 5 and then given an order of 4, it read 0. It now holds the level's running total at zero, so the same level reads 4, and it warns with the level and the first event. audit counts those rows as negative_level (a warning). The bundled sample, the toy session and a synthetic session have none.
The book replay keeps its click script in a notebook (#121). IPython displayed a BookReplayFigure through Plotly's own renderer, which drops the script that makes trades clickable, and the tutorial build dropped the figure entirely. The figure now displays as HTML in a notebook, as documented.
Stale orders and crossing are measured correctly on display tables. display_result converts sizes to base-asset floats. price_level_volume summed those floats, and a level whose last order left could keep a remainder such as 3e-17 in place of 0. The level then stayed in the faithful book. On the bundled sample, 22,915 depth rows kept a remainder. detect_stale_orders reported a touch_seconds of 0.0 for the stale ask at 78,333, which holds the touch for 1,645.7 s, and data_quality_summary reported the book crossed 92.03% of the time against 91.61%. price_level_volume now sums float sizes that sit on a decimal grid as whole multiples of that grid, the way it sums integer lots, so the display tables give the same results as the pipeline's own tables. Float sizes on no decimal grid, such as a size worked out by float subtraction, are still summed as floats, and price_level_volume now warns when it does that.
The price-level depth is replayed in event order. price_level_volume and depth_metrics sorted rows on the timestamp alone, so the rows of one instant were replayed bids first, then asks, by price, and a depth_summary row was not always the book after the event it names. On the bundled Bitstamp sample, for 44,043 of 156,719 new limit orders the last row before the order was replayed after it, so "the book before the order" already held the order. Both now use the order the per-order rebuild uses (schemas.time_order_keys: the timestamp, then sequence, event_id and ingest_seq where present). Each level's running total is also taken in that order, so every depth row holds the level's volume after its own event, and depth carries the sequence and ingest_seq columns over from the events when they have them. The book replay (price_level_snapshots) uses the same order, so its touch still equals the depth summary's. On the sample, the book at the end of an instant changes at 2,634 of 130,244 instants (the best bid or ask at 2,453), because removing crossed levels depends on the order rows arrive in, and aggressiveness_bps changes on 28,898 of 314,057 events. On a synthetic session, which is never crossed, every event's last depth_summary row now matches the per-order rebuild.
aggressiveness_bps is measured against the book at the order's own time. order_aggressiveness found the book standing before each order by taking the depth_summary row with the next-lower event_id. That is only correct when event ids are in time order. The Bitstamp loader numbers events after sorting them by order id, so on Bitstamp that row could come from any time in the session. On the bundled sample, 51,805 of 156,718 new orders were measured against a book from after the order, and 7,399 against a book more than a minute away, up to 30 minutes. The lookup now uses the documented event order (time_order_keys): the last depth_summary row with an earlier timestamp, or with the same timestamp and a lower event_id. aggressiveness_bps changes on 45,087 of the sample's 314,057 event rows (14.4%). Databento, LOBSTER and the synthetic generator number events in time order, so their values do not change.
compute_vpin refuses a bucket size in the wrong units. Sizes are integer lots, so bucket_volume=5.0 on the Bitstamp sample means 5e-8 BTC and asked for about 300 million buckets, enough to run the machine out of memory. compute_vpin and bulk_volume_classification now raise ValueError when a bucket would make more than MAX_VOLUME_BUCKETS (one million) buckets, and say which units bucket_volume is in. The how-to pages that passed bucket_volume=5.0 now size the bucket from the trades.
Docs written before lots and UTC timestamps are corrected. The schema page gives the current version, 4.0. The step-by-step and synthetic-data pages say timestamps are tz-aware UTC. Tutorial chapter 3's LOBSTER round trip now passes lot_size=1.0; without it every size was written as a fraction of a lot, and the round trip read back no executed volume.
The tutorial pages show no warnings (#120). Chapter 3 now reads its LOBSTER files back with the toy's tick size. With LOBSTER's one-cent default, $99 came back as 9900 ticks against the original 99, so the books before and after the round trip did not match. An empty book snapshot no longer draws an empty legend. The docs build now fails when a tutorial cell prints a warning or a figure fails to draw, and when a gallery recipe raises a warning.
An Independent Reserve capture no longer keeps orders cancelled before it started (#315). cryptofeed takes the opening book from the venue's REST interface when the first stream message arrives. The venue serves that book from a cache, and its orders are older than its CreatedTimestampUtc, so the book could be a few seconds older than the stream. An order cancelled in between stayed in the book until the capture ended, and a trade at a price past it made the book crossed: 13 orders in a 12-minute capture, two of them at the best ask. The cryptofeed source now uses its own Independent Reserve feed, which fetches the book again, once a second, until it was created at least 3 s after the first message. The stream waits meanwhile and loses nothing. After 15 tries it uses the last book and logs a warning.
Coinbase captures through ccxt record the taker's side (#308). Coinbase reports each trade with the side of the maker, and ccxt passes it on unchanged, so every Coinbase capture had its trade signs reversed, and so did order flow imbalance, VPIN and everything else built on them. The ccxt source now reverses the side for the coinbase venue, on both its websocket and REST trades, and records trade_side_reversed in meta.json. A Coinbase capture without that key was made before this change: reverse its signs. coinbaseexchange needs no change, because ccxt already reverses its side.
An Independent Reserve order no longer stays in the book after its cancel (#313). Before 3.0, cryptofeed forgets an Independent Reserve order after its first change, even when the order still has size, and skips every later message for it. So the cancel after a partial fill never reached the capture, the order stayed in the book until the capture ended, and audit reported it as a stale resting order. The cryptofeed source now uses a subclass of cryptofeed's Independent Reserve feed that keeps the order while it rests. cryptofeed 3.0 fixes this too, but needs Python 3.13.
audit no longer stops on UUID order ids (#311). A stale resting order from an Independent Reserve capture made audit raise ValueError. StaleOrder.id now holds the id as the feed writes it (int | str), and the text and JSON reports print it.
cryptofeed's Independent Reserve trades are linked to their orders (#311). The trades now carry both order ids (BidGuid and OfferGuid) in buy_order_id and sell_order_id, and each fill is recorded from the trade, as on Bitstamp. On a per-change book, a fill that the trade and the book both report is recorded once, and a book message that repeats an order's size writes no row.
An Independent Reserve capture's trades.csv no longer holds other markets' trades (#316). The venue sends a market's trade channel the trades of its other markets for the same coin, so a BTC-AUD capture also got BTC-NZD and BTC-SGD trades, priced in NZD and SGD. audit read them as trades through the book and reported resting orders as stale. The cryptofeed source now writes a trade to trades.csv only when it is for the capture's pair, and counts the others as other_market_trades in meta.json. The venue keeps one book for all its markets, so a trade left out still reports the fill of any order it names in the capture, at the order's price.
audit no longer reports lost messages on cryptofeed captures that lost none (#309). On Bitfinex and Blockchain.com cryptofeed's sequence number counts every message on the connection, trades and heartbeats too, and on Independent Reserve cryptofeed passes on no change to an order it does not hold. So the book rows skip numbers, and audit reported each skip as a sequence_gaps error. The cryptofeed source now declares sequence_kind = SequenceKind.MONOTONIC and records it in meta.json, so audit checks only that the number never goes back. meta.json reports sequence_out_of_order in place of sequence_gaps and sequence_missing, so a long capture's manifest no longer counts the skips as dropped messages. A capture whose meta.json records no sequence_kind, such as one made before this change, is checked the way the source that made it declares now, not the way --source does.
A quiet book no longer gives a blank depth heatmap (#303). The heatmap leaves out each price level that does not change in the window. When no level changed, as in a short capture of a quiet prediction market, that left out every level and drew blank axes, with only Not enough data for any price level in the log. Now the heatmap draws every level in that case, with a note under the title that no level changed. When the heatmap really is empty, the note says why: no depth data, no level between two named prices, no level in the chosen volume range, or no resting orders in the time window. All three backends put the note on its own line under the title and write it to the log. An empty heatmap still draws the midprice and trades. A spread or trades frame with no rows in the window no longer sets a NaN price range that left out every level. The gallery's depth-heatmap card names plot_result(result, "depth_heatmap", show_all_depth=True), and the Kalshi and Polymarket how-tos describe the case. The matplotlib heatmap also no longer goes blank when one level has a single row in the window.
The gallery prepares each card's data once (#303). The backend columns of a card now share one prepared payload, so the prepare step runs, and logs, once per card instead of once per backend.
The clock checks compare two real clocks, and run on L2 (#310). Where a venue sent no time, the cryptofeed and ccxt sources wrote the time they handled the message into exchange_timestamp, which is later than the receive time. On cryptofeed's Bitfinex and Blockchain.com books, and on the Independent Reserve opening book, audit then failed the capture on exchange_time_after_receive. Such rows now copy the receive time, as LOBSTER's one clock fills both columns. The new Clocks declaration (BOTH, RECEIVE_ONLY or VENUE_ONLY) says which clocks a source's rows carry. LOBSTER declares VENUE_ONLY. A live capture records clocks and books_without_venue_time in meta.json; read it with recorded_clocks. With one clock, audit does not run the two clock checks and says why. The L2 depth frame now keeps exchange_timestamp when depth.csv has it, so the clock checks run on L2 captures, which before were never checked. The clock checks leave out the opening book's rows (origin snapshot), whose clocks the capture may not have measured; the L2 depth frame now keeps origin for this.
A cryptofeed REST opening book replays before the message that changed it (#310). On Binance and Independent Reserve, cryptofeed fetches the opening book while it handles the first live message, and hands the book over first with a later receipt time. On Binance (L2), replay applied the message and then the book's older size. On Independent Reserve (L3), with sequences tracked, the numbered message sorted before the book. The capture now holds a book with no delta until the next one arrives, and places it 1 ms before a message received earlier. The message keeps its own receipt time, and the book's rows are marked as the opening book.
A Bitstamp capture no longer keeps trades from before its snapshot (#301). It already skipped order messages from before the REST snapshot, but kept the trades from the same time. The orders those trades filled are not in the capture, so audit counted the trades as unmatched: up to 52% of a segment's trades with 30-second segments. The trades are now skipped and counted as pre_snapshot_trades_skipped in meta.json. At a roll, the previous segment already has these trades.
raw.jsonl no longer stops a cryptofeed capture of independent_reserve or blockchain. cryptofeed reads these venues' date and time strings as datetime, date and time values, which raw.jsonl could not write, so every segment failed unless --no-raw was passed. They are now written as ISO 8601 strings. Any other value or dict key that JSON cannot hold is written as text (str(value)), and a frame JSON cannot hold at all, such as one that refers to itself, is skipped. Neither stops the capture. Each segment's meta.json names the types written as text (raw_text_types) and counts the skipped frames (n_raw_frames_skipped); the manifest adds up raw_frames_skipped, and the capture logs each kind of warning once.
A custom plot keeps working after it is added to the gallery (#302). The "A new plot" guide registered a plot at Level.L2, then registered it again at None to put it in the gallery. After the second step, plot("cumvol") raised and told you to pass Level.L3, which was not registered. The guide now registers its example once, at None, draws the gallery on the one backend it registered, and shows how a plot drawn at a level goes into the gallery as a PlotConcept. A test runs the guide's plot section as written. See Changed for the rule that replaces the second registration.
A gallery card added the wrong way says how to fix it (#312). A level-less plot added to GalleryModel.concepts, a plot drawn at a level added to GalleryModel.analytics, or a PlotConcept variant at a level no renderer is registered at, showed a bare "Not available", the same as a renderer that raised. The gallery now checks each card against the levels its renderers are registered at. When they disagree, the card says how the plot is registered and how to change the model, and the log has the same text once. A backend that cannot be loaded says so on its cards, and is tried once instead of once for each card. A backend with no renderer for a plot that other backends draw still shows a bare "Not available".
On Python 3.11, a capture stops each segment when asked (#296). The live sources waited for messages with asyncio.wait_for, which on Python 3.11 can drop a cancel that arrives with a message. A roll then left the old segment streaming next to the new one to the end of the capture, with no further rolls, and SIGTERM or Ctrl-C did nothing. The sources now use asyncio.timeout, and ruff bans asyncio.wait_for in this repository. The runner cancels a stream again while it keeps yielding items, so a plug-in source with the same pattern cannot block a stop; a stream that is closing its connection is left to finish. Python 3.12 and later were not affected.
A segment that does not stop cannot hold up the capture (#296). A segment asked to stop has 20 seconds to close. After that it is cancelled and closed from its files, like a segment a crash left open. The manifest keeps why it was stopped and records the delay as its error, not as a gap. If closing its files fails, the error says so and the capture carries on. A segment covers the market only until it is asked to stop, and one asked to stop before its first live event covers nothing, so it no longer adds a second gap next to the real one. At the end of a capture all running segments are stopped together, so SIGTERM ends a capture in under a minute even when a source hangs.
audit no longer fails a complete Databento file (#298). Databento numbers every message on the venue's channel, so the numbers in one instrument's events skip the other instruments' messages and the trade and fill records, which become trades. audit read every skip as a lost message and failed a sound file with a sequence_gaps error. DatabentoSource now declares sequence_kind = SequenceKind.MONOTONIC, so audit checks only that the numbers never go back. audit reads a source's declared sequence_kind when no capture meta.json records one; read it with the new sequence_kind_of. recorded_sequence_kind returns None when the capture records none, as recorded_feed_type does.
A Bitstamp L3 capture through cryptofeed passes audit (#284). cryptofeed's Bitstamp L3 channel is detail_order_book: a picture of the top 100 bids and top 100 asks about 10 times a second, not every order. Four defects followed from that:
An order that dropped past the 100th place was recorded as deleted, and as created again when it came back. It now stays tracked at its last size until the book shows its price again. This removes the duplicate created ids, and the clock errors the Bitstamp reader made from them.
Book rows carried the venue's time in both clock columns. timestamp is now the time the capture received the message, like the trades.
Trades dropped the maker and taker order ids that Bitstamp sends. They are now read from the raw message.
A fill showed only as a size change between two pictures, so a fully filled order read as cancelled. A trade that names a tracked order now reports its fill as it arrives, and a filled order is deleted at size 0, as the native Bitstamp feed reports it. A picture that shows an order gone before its trade arrives holds the delete for up to 2 seconds for it.

On a 90-second capture, makers now link on 40 of 42 trades, up from 16. For analysis, the native bitstamp source still shows more: every order, takers included.

A capture that fails now exits non-zero and says why (#283). ob-analytics capture used to exit 0 and write "errors": 0 to meta.json when the stream raised, or when the source's optional extra was missing. A source is now checked before the capture starts: a missing extra, or an unknown ccxt or cryptofeed venue, stops the run with status 1 and creates no output directory. An error in the snapshot, the stream or the shutdown events keeps the rows already written, is recorded in meta.json as capture_error and capture_error_phase and counted in errors, and makes the command exit 1. A source can take part in the early check by adding a preflight() method (SupportsPreflight).
A Binance capture no longer loses price levels near the top of the book (#101). ccxt deletes the levels of a Binance book that fall past the depth it is given, and Binance sends a level again only when it changes. After the price moved away and back, the captured book had holes in its top 20 levels. The ccxt source now asks for the whole Binance book (see #275 below for what it does with that book) and never gives ccxt less than 1,000 levels a side to track, so a level rarely falls out of what ccxt itself knows. ccxt's opening snapshot is now as deep as --depth-limit (never less than 1,000 levels), so a Binance capture can record up to the 5,000 levels a side that Binance sends, about 1% from the price. A deeper --depth-limit is refused.
audit no longer fails every ccxt capture with missing sequence numbers (#101). The ccxt nonce only rises: a Binance diff covers a range of update IDs, and ccxt can apply several diffs before it returns a book. A capture now records "sequence_kind": "monotonic" in meta.json, and audit then checks only that the number never goes back. Captures made before this change record no kind and are still read as contiguous.
ccxt book rows are stamped with the time they arrived (#101). The timestamp of a ccxt depth.csv row was the venue's book time, while its trades used the receive time. ccxt stamps its first Binance book with its own snapshot's time and then applies older diffs, so sorting on that time swapped two updates and left a stale level: audit reported the book crossed for 90% of a session. timestamp is now the receive time, and the venue's time is kept in a new exchange_timestamp column.
A ccxt book that loses sync is fetched again (#101). When ccxt finds a missing Binance diff it drops its book and raises an error, which used to end the book for the rest of the capture while trades went on. The capture now asks for the book again (up to 10 times) and counts it as book_resyncs in meta.json.
A venue that refuses your location gives a one-line error. Binance answers HTTP 451 from some countries. capture ccxt now says so and names --exchange binanceus and --market-data-mirror, instead of printing a traceback.
A level that only left --depth-limit, not the book, is no longer recorded as cancelled (#275). Every ccxt capture used to crop depth.csv/raw.jsonl to the top --depth-limit levels a side, so a level that was still resting just outside that crop read as a 0 row when the price moved, the same as a real cancel — 28% of the removals in a five-minute Binance BTC/USDT capture. The capture now records whatever ccxt reports, uncropped: Coinbase, Bitstamp and OKX ignore --depth-limit and always hand ccxt their whole book (over 20,000 levels a side on Coinbase); Binance and its family are asked for their whole book too (see #101 above). Kraken is unaffected, because it subscribes at exactly --depth-limit levels and drops a level from its own book once the price moves it out of that window — a 0 row there can still be either a real cancel or Kraken's own window exit, which is inherent to how Kraken reports its book. docs/howto/ccxt.md now says what a 0 row means, per venue.
compute_vpin now warns when its default bucket size is too big for the capture (#274). The default bucket_volume (average daily volume ÷ 50) scales a short capture up to a full day, so the bundled sample fills only one bucket and the trailing vpin_avg never covers a full window. That was already recorded in attrs["diagnostics"], easy to miss on a frame that otherwise looks fine — compute_vpin now also raises a UserWarning when there are fewer complete buckets than n_buckets, naming a smaller bucket_volume or n_buckets as the fix. The flow-toxicity how-to gains a short-captures section with a working example on the sample.

Changed¶

The price-level depth follows an order that moves or grows. price_level_volume used to count every one of an order's rows on the price of its first row, and ignored a rise in its size. It now reads a changed row that reports no execution and carries a new price as a move: the order's volume leaves the old level and joins the new one. A changed row that reports no execution and a larger size adds the difference where the order rests. Deletes and rows that report an execution are still taken off where the order rests, whatever price they carry, so the fix for Bitstamp's deletes at the wrong price is kept. The price-level rebuild now agrees with the per-order rebuild (book_state) on these orders. This matters for Databento, whose modify can move an order or make it bigger. The Bitstamp and LOBSTER outputs are unchanged: neither feed moves or grows an order this way.
A trade the venue left unlabelled is now classified on the L3 path too. Pipeline.run labels any trade with no aggressor side against the reconstructed quotes, filling one subset at a time instead of all or nothing, and never overwriting a side the venue did state. This was already what the price-level path did; it now also covers a per-order feed that states the aggressor on most trades but not all, which is what Databento does for auctions, non-displayed orders and off-exchange prints. No change for a feed that labels every trade, which is every other source in the package.

Added¶

A Bokeh plot backend (#123). result.plot(concept, backend="bokeh") renders the core concepts — trade_tape, depth_heatmap, book_snapshot, depth_chart — as interactive Bokeh figures, alongside the static Matplotlib default and the Plotly backend. Suited to Bokeh / Panel server dashboards and streaming views. Ships in the new bokeh extra: pip install "ob-analytics[bokeh]".
Binance venue notes and a market-data mirror (#101). A new how-to page covers capturing Binance spot through ccxt: the location block, the binanceus alternative, the depth a 100-level book reaches, and trade sides. capture ccxt --market-data-mirror (CcxtSettings.market_data_mirror) reads Binance spot from Binance's market-data-only endpoints. New SequenceKind and recorded_sequence_kind(), a kind argument on detect_sequence_gaps(), and a sequence_kind argument on data_quality_summary().
Find hidden liquidity (#111). detect_icebergs(events, trades) finds iceberg orders from their refills: a resting order filled out, then a new order at the same side and price within one millisecond. It chains refills into one suspected iceberg and gives each a confidence. hidden_trades(events, trades, depth_summary) returns the trades that printed strictly inside the visible spread. On one day of LOBSTER AAPL it finds 85% of the type-5 hidden executions, with no false ones. The synthetic generator now labels its iceberg slices in SynthSession.icebergs, so the detector can be scored exactly. See the hidden liquidity how-to.
Iceberg refills and hidden trades drawn on the depth heatmap and order activity map (#272). Both L3 faces now overlay detect_icebergs and hidden_trades: a diamond marks each refill, joined by a line per iceberg (opacity = confidence); a star marks a hidden trade, with a thin line to the standing best bid and best ask so the print reads as inside that spread. A filled star is a confirmed hidden order; an open star is a trade to check, where the maker order was actually visible or its maker identity did not resolve at all — the diff-feed case the how-to guide describes. Both overlays clip to the gallery's zoom window, and are absent without error when a run has neither. New prepare.hidden_liquidity_overlay builds the same overlay for a custom plot. hidden_trades now returns best_bid_price/best_ask_price in depth_summary's own dtype instead of always casting to int64, so a caller already holding display-unit floats gets floats back rather than a silently truncated value.
A feature table for models (#149). features(trades, quotes) returns one tidy table: a point in time on each row and a microstructure feature in each column — the shape a model or a study wants. It replaces a join per measurement.

Two decisions make the table, and they are separate. Where the rows fall is a bar rule, so features() takes the same sampling arguments bars() does and the same arguments give the same cut in both; a time grid is the time rule. What each column measures is a Feature, and ten ship: price, returns, flow, spread, mid_price, micro_price, imbalance, depth, vpin and kyle_lambda, writing 20 columns between them. register_feature adds one of your own, usable by name with no edit to the package.

Every row is stated as of the close of its bar. The trade columns hold what happened inside the bar, and the book columns hold the book as it stood at the close, a backward as-of join. Nothing from after that instant reaches the row, so the table carries no look-ahead — which is tested by truncating the inputs and checking that the rows that survive are unchanged, for every rule and every feature. The table holds no target either: a target looks forward, and building one is a shift the caller makes deliberately.

Two quote states are not books anything could have traded against, and both would otherwise arrive as ordinary numbers: a side with nothing resting on it, which the depth engine marks with a price of 0, and a crossed book, which a diff feed can genuinely hold. readable_quotes() drops them from the reference series, so a row reaches back to the last quote it could read — the same test transaction_costs already applied before measuring against a mid. A locked book, bid equal to ask, is a real state at a spread of zero and is kept.

Without a quotes frame the five book features are skipped and the table holds the trade features alone; naming one explicitly raises instead. Two features that would write the same column are an error rather than a silent overwrite, and so is a name listed twice in include; a trailing window a feature cannot use is refused when the feature is built. See the "Build a feature table" how-to, which ends in a baseline model.

Databento market-by-order files (#100). DatabentoSource reads Databento's DBN files in the MBO schema, which is a per-order feed: every record carries an order id, so a file replays through the full L3 path with order lifetimes, queue position and order classification. It reaches many venues that no other source in the package does, US equities and futures among them.

Databento reports an execution as a fill record that does not change the book, followed by the cancel or modify that takes the size off it. The loader pairs the two, so fill tells an execution apart from a cancel the trader asked for, and each trade names the resting order it hit. The aggressor's side comes from the venue rather than a classifier. A book clear deletes the orders still resting, and the record's two clocks are kept apart: Databento's receive time orders the events, the venue's own becomes exchange_timestamp.

A modify that moves an order to another price or makes it bigger is recorded as a changed event with the new price and size, and the depth follows it (see Changed). The loss of queue priority is not modelled. The one modify the depth still cannot follow is one that carries a fill and also moves the order or changes its size by more than the fill; the loader says how many rows the depth will be off by.

A feed the loader does not understand is refused: a publisher that only sends top-of-book or price-level data, because its order ids mean nothing; a price-level schema; a file covering more than one book; an action outside DBN's own alphabet; an order id too big for the schema's signed 64-bit id. A malformed record inside a feed it does understand — no price, or no side on a book action — is dropped and counted in a warning, because refusing a whole session over a handful of them would be worse.

Trades are built from the fill records by default, so each names the resting order it hit. A trade the publisher sent no fill for — an auction, a trade against a non-displayed order, an off-exchange print — is then left out, and the loader warns with the volume. DatabentoSettings(trades_from="prints") builds the trades from the whole tape instead, without makers; use it for VWAP, bars, flow toxicity and costs. DatabentoWriter writes an events frame back out as DBN. scripts/databento_window.py sizes a query against the in-memory envelope before downloading it, then runs one window at a time. databento is an optional extra (pip install "ob-analytics[databento]"). See the "Process Databento MBO files" how-to.

Flow-toxicity results say when they rest on too little data (#119). VPIN and Kyle's λ were designed for markets that trade thousands of times a minute, and on a thin tape they used to return a number with no sign that it meant little. KyleLambdaResult now has significant and diagnostics: λ is flagged when there are fewer than 30 regression windows (KYLE_MIN_WINDOWS), when |t| is below 2 (KYLE_MIN_T_STAT), or when the fit is undefined. It also carries a confidence interval, ci_low / ci_high, from a block bootstrap over the windows (1000 resamples by default, under a millisecond on the bundled sample, seeded with seed=0 so the same trades give the same interval; n_boot=0 skips it).

compute_vpin records how it ran in the frame's attrs: bucket_volume, bucket_volume_rule, n_buckets, and diagnostics, which flags a result with fewer complete buckets than n_buckets. bucket_volume is now optional. Left out, it is picked by the new vpin_bucket_volume(trades), which applies the common rule of average daily volume ÷ 50; a session shorter than a day is scaled up to a day at the rate it traded (trading_day="24h" by default). Existing calls are unchanged.

On the bundled sample, λ at 5-minute windows is flagged on both counts (7 windows, t = 1.45) and its interval spans zero; VPIN with the default bucket fills one bucket and says so. Tutorial chapter 6 and the flow-toxicity how-to now show these checks in place of the hand-written caveats.

Bars: the trade stream resampled into OHLCV rows (#148). bars(trades, rule, threshold) cuts a trades frame into bars and returns one row each with open, high, low, close, volume, turnover, VWAP, and the buy/sell split of that volume. Five rules ship with it: time (a fixed span of the clock), tick (a fixed number of trades), volume and dollar (a fixed amount of size, or of price × size), and imbalance (signed size drifting a set amount from where the bar opened). Leave the threshold out and the rule picks one aiming at about 50 bars.

What differs between bar types is only where the boundaries fall, so that is all a rule decides: BarRule states a name, how it reads its threshold, and which bar each trade belongs to. register_bar_rule adds one of your own, usable by name with no edit to the package. Feeds that don't label the aggressor are classified the same way the flow-toxicity metrics do.

Bars draw as a "bars" plot face on both backends — candles over a strip of volume coloured by the net aggressor — and bars_panel() puts them in a gallery. The demos show a clock cut and a volume cut side by side. See the "Build bars from trades" how-to.

Transaction cost and price impact (#110). A new cost module answers what trading cost, rather than what the book advertised. transaction_costs(trades, quotes) returns one row per trade with the effective spread — what the taker paid to cross — split into the realized spread the liquidity provider kept and the price impact the trade caused, in price units and in basis points. The three add up exactly, trade by trade. cost_summary() reduces that to volume-weighted session figures and reports how many trades each one could be measured on.

amihud() and roll_spread() read liquidity from the trade prices alone, so they run on a tape with no quotes and no aggressor side: the price move a unit of turnover buys, and the spread implied by bid-ask bounce. Roll also returns the lag-1 autocorrelation of the price changes, which its model puts at exactly -0.5; how far the number sits from that is how little of the price movement the bounce explains. On the bundled capture it is +0.197 and the estimate has no real root, so it is NaN rather than a number the model does not support. The diagnostic matters in the other direction too: when the autocovariance lands negative by chance Roll returns a spread that is not there, and the autocorrelation is what catches it.

The mid a trade is measured against is the last quote strictly before it, skipping crossed quotes: on a frame built from the same event stream, the quote sharing a trade's instant is the book after that trade took the touch, and a crossed book has no midpoint at all. A trade in the last horizon of the capture has no future mid, so its realized spread is NaN rather than the final quote reused.

transaction_costs takes mid_column to measure against a reference other than the plain mid — "micro_price" for the size-weighted mid, which on the bundled capture reads 1.24 bps against the plain mid's 1.45.

The decomposition draws as a level-less transaction_costs face on both backends — two lines with the impact as the band between them — and transaction_costs_panel() puts it in a gallery. Both demos now include it. See the "Measure transaction costs" how-to.

Polymarket prediction markets, through the ccxt source (#103). ob-analytics capture ccxt --exchange polymarket --pair <token id> streams one outcome's order book and trades over Polymarket's public websocket, with no account or API key. --pair is Polymarket's token id for the outcome, which the Gamma API lists as clobTokenIds. Each outcome is its own book, and its trades are priced in that outcome.

Polymarket makes a market's tick finer as the price nears 0 or 1, so a ccxt capture now makes its recorded tick size finer when a price arrives between two ticks, and counts each change in tick_size_changes in meta.json. The replay then reads every price exactly. See the "Capture Polymarket prediction markets" how-to.

Kalshi prediction markets, through the ccxt source (#102). ob-analytics capture ccxt --exchange kalshi --pair <market ticker> records a Kalshi market's order book and trades from Kalshi's public API, with no account or API key, and ob-analytics process replays it through the L2 path. The ccxt source now looks up CCXT's prediction markets (ccxt.prediction: Kalshi, Polymarket and others) as well as its crypto exchanges; before, --exchange kalshi failed with "Unknown CCXT exchange". binance and hyperliquid are in both lists, so the plain id keeps meaning the crypto exchange and prediction/<id> picks the prediction market.

The captured book is the market's Yes book: a bid to buy No at p is recorded as an offer to sell Yes at 1 - p, and a trade is priced in Yes and signed from the Yes side. A ccxt capture also records its market's tick size in meta.json, which process and audit use. See the "Capture Kalshi prediction markets" how-to.

A capture records which rows came from its opening snapshot (#237). orders.csv and depth.csv gain an origin column: snapshot for the opening book, stream for a live message, shutdown for a synthetic close-out. The capture runner fills it in, so every live source gets it without a change, and the loaders carry it through to events. Before this, the only way to tell a snapshot row from a live one was to compare its exchange_timestamp with snapshot_microtimestamp in meta.json.

meta.json also reports n_snapshot_unconfirmed: how many orders in the opening book no later order event or trade mentioned. The bundled Bitstamp sample has 6,294 of 6,512. Almost all of them sit far from the touch and did not trade, but two stale asks among them held the best ask for most of the session. See "Capture live data".

audit names stale resting orders (#234). A trade above a resting ask, or below a resting bid, shows that the order has gone. An order the venue then does not report again within one second is now reported as a stale_orders warning, and the worst one is named with its id, side, price and how long it held the touch. On the bundled Bitstamp sample this names ask 2002347646152704, which held the ask touch for 27 minutes and causes almost all of the 91.6% crossed time. The crossing note no longer calls a diff feed's crossing normal when the run has stale orders. Nothing is removed: order_book() still replays what the feed said. New public names: detect_stale_orders, StaleOrder, DataQualitySummary.stale_orders, and a tick_size= argument on data_quality_summary.
A metric registry, so a user metric runs and plots with no core edit (#140). A metric is a plain object with a name, a title, the levels it applies to, compute(result) and prepare(frame) — no base class to inherit, the same structural typing sources and writers use. Register it with register_metric(metric), or ship it in your own package under the ob_analytics.metrics entry-point group and load_metric_plugins() finds it at import ob_analytics.

A registered metric is a level-less plot concept under its own name, so a renderer at (name, None, backend) is its face. It then appears in available_concepts(result), renders through result.plot(name), and gets its own gallery card with no extra_panels=. Metrics run when asked for, not during Pipeline.run: result.metric(name) computes one and result.metrics() computes every metric whose levels include the run's resolution — so an L3-only metric is skipped on an L2 run instead of failing on its empty events table, and a metric that raises is logged and its card dropped, so one broken metric cannot stop the gallery being built. New public names: Metric, register_metric, list_metrics, get_metric, load_metric_plugins, PipelineResult.metric / .metrics. See the "A new metric" how-to.

ob-analytics audit, a data-quality gate (#108). The old validate verb is now audit (the old name still works), it scores the run against named checks, and it exits non-zero when one fails — so a script can stop before trusting a feed. Five checks are new: orphan orders (changed or deleted with no created row), non-positive prices, negative volumes or fills, and the two clock-order defects — a venue timestamp later than the receive timestamp, and messages that arrived out of venue order. audit also loads with track_sequence on, so the dropped-message check (#146) reads a venue sequence whenever the feed carries one.

Each check carries a Severity: an error fails the run, a warning fails it only under --strict, and info never does. A crossed resting book is scored by feed type, not by size — an error on a matched book, a faithful replay on a diff feed. --json emits every check plus an ok verdict; --from-parquet audits a saved process output without re-running the pipeline. New public names: Severity, QualityCheck, and DataQualitySummary.ok / .errors / .warnings / .checks. See the "Check data quality with audit" how-to.

PipelineResult.to_arrow() and PipelineResult.to_polars() (#104). Both return the run's four tables — events, trades, depth, depth_summary — keyed by name, with the same keys on every run: on an L2 run events is an empty table, not a missing key. The Arrow tables carry the schema version and tick size in their metadata, the same key-value metadata the Parquet files carry, so a reader handed tables in memory is no worse off than one reading files. Polars is not a dependency and is not installed; to_polars() raises ImportError with an install hint when it is missing, and Polars keeps no schema metadata, so the version and tick size do not survive that conversion.
The frame-type contract is written down in Frame types: pandas in, pandas out: public functions take and return pandas, plug-ins are handed pandas, and the versioned Parquet is how other tools read the output. The reasoning is in adr/0002-dataframe-library.md.
cryptofeed source for live L2 and L3 capture (ob-analytics capture cryptofeed --exchange <venue> --pair <symbol>). The per-order complement to the CCXT source: venues publishing an order-by-order book record orders.csv with the venue's own ids and replay through the full reconstruction pipeline; the rest record depth.csv for the L2 path. The level is discovered from the venue's declared channels rather than a hardcoded list, and --level forces it — except L3 on a venue that publishes none, which raises. Ships as the optional [cryptofeed] extra, imported lazily. See the new "Capture cryptofeed venues" how-to.
sequence is now written to orders.csv. The capture sink dropped the venue sequence on the L3 path, so detect_sequence_gaps had nothing to read; L2 already kept it. cryptofeed captures also report sequence_gaps / sequence_missing in meta.json.
L2 (price-level) depth-native ingestion path. Price-level feeds (Binance, Kalshi, Polymarket, most CCXT sources) publish [price, quantity] levels and diffs with no order IDs; ob-analytics now ingests them as a first-class L2 resolution instead of faking per-order state. A format declares its resolution (Level.L2 / Level.L3, exposed as ob_analytics.Level); an L2 format's loader is a DepthSource that yields the depth frame directly, and Pipeline.run takes the price-level path — depth metrics / spread and trade-sign classification run, while the per-order stages (set_order_types, order_aggressiveness, queue reconstruction) are skipped. PipelineResult gains a resolution field and, on an L2 run, returns an empty (schema-valid) events frame. Ships the depth_csv format (L2DepthLoader, L2TradeReader, DepthCsvWriter, DepthCsvFormat) for the canonical L2 CSV schema, a toy_l2_depth() / toy_l2_trades() synthetic snapshot+delta fixture, and data_quality_summary + the gallery degrade gracefully (L3-only faces skipped, not errored). ob-analytics process|validate --format depth_csv works from the CLI. Unblocks the aggregated venue connectors. Documented in a new "Process L2 feeds" how-to.
Trade-sign classification (ob_analytics.trade_sign) for feeds that don't label the aggressor side. tick_rule (last-price-change sign), lee_ready (quote-midpoint test with a tick-rule fallback), and bulk_volume_classification (BVC — the buy fraction of a volume bar via the standardized-price-change normal CDF). classify_trade_sign(trades, method=..., quotes=...) is the per-trade entry point. compute_vpin and order_flow_imbalance gain sign_method / quotes arguments and now synthesize direction automatically when the trades frame has none — so VPIN and OFI run on L2 / aggregated captures, not just L3. A native direction is still honored unchanged (sign_method=None). On the bundled Bitstamp L3 sample the classifiers agree with the true maker/taker side ~0.83 (tick) / ~0.79 (Lee–Ready) — validated by a test harness.
Feed classification. Every format declares a FeedType (matched_book vs diff_feed) through a feed_type attribute — BitstampFormat → diff_feed, LobsterFormat → matched_book — so downstream code reasons about crossed books by coordinate, not by format name. Exposed as ob_analytics.FeedType.
order_book(..., uncross=True) evicts crossed resting orders for display, mirroring the depth engine's crossed-level eviction. The default stays faithful, so a diff feed's genuinely crossed resting orders are replayed as-is. Threaded through prepare.book_snapshot(..., uncross=True) (also drives depth_chart) and available frame-level as analytics.uncross_book_sides.
Per-run data-quality summary. data_quality_summary() and the new ob-analytics validate <source> CLI verb report the crossed-resting %, unmatched-trades %, duplicate ids, and pre-existing-order count. A new "Data quality: matched book vs diff feed" explanation page and a validate how-to document the distinction.

Fixed¶

The VPIN chart draws an empty panel when no bucket is complete. A capture with less volume than one bucket gives compute_vpin zero rows, and plot("vpin", ...) then raised: a TypeError with matplotlib, a KeyError with both backends when the frame had no columns. Both backends now draw the axes with "(no complete buckets)" in the title. compute_vpin now returns its usual columns and dtypes when it has no rows, where before it returned a frame with no columns (or, with sign_method="bvc", columns of object dtype).
A price-level file no longer has its prices rounded to the tick size. L2DepthLoader and L2TradeReader converted each price to the nearest whole number of ticks, so a price finer than tick_size moved without a warning: a Kalshi price of 0.036 loaded as 0.04 at the default 0.01 tick, and the most traded Kalshi markets quote in tenths of a cent. Both now raise ConfigError when a price is not a whole number of ticks, and say which tick size to set. A ccxt capture records its market's tick size in meta.json, and ob-analytics process and ob-analytics audit read it from there (recorded_tick_size), so a CLI replay needs no extra option.
A streamed ccxt capture no longer writes a trade twice. ccxt's Polymarket websocket handed back a trade it had already delivered, together with the next new one, and only a polled capture skipped repeats. Every ccxt capture now skips a trade identical to one it has written: same id, time, price, size and side. Two fills that share a Polymarket id (the settling transaction) are both kept. meta.json counts the skipped repeats in duplicate_trades.
A polled ccxt capture no longer records trades from before it started. On a venue without websockets, the first poll of the trade tape returns the venue's recent history, which on Kalshi reached back nine hours. Those trades were written with the capture's receive time, as if they had just happened. The capture now drops trades older than its opening book.
A Bitstamp capture no longer starts from a snapshot older than its stream (#237). The capturer subscribes to the WebSocket, then fetches the REST book. It assumed the stream already covered the moment the book describes, but it often does not: in a live test the first order message came 0.7 s after the snapshot's microtimestamp, and the bundled sample shows the same 0.74 s gap. An order deleted in that gap stayed in the capture until the synthetic deleted at shutdown. In the bundled sample, one such ask was the best ask for 89% of the session.

The capturer now fetches the book again, a second apart and up to 10 times, until some buffered order message is at or before the snapshot's microtimestamp. meta.json gains snapshot_fetches and snapshot_overlap. In the live test, the second fetch no longer listed any of the 15 orders that were gone. Eight of those were orders that trades printed through.

A price level now empties when the order resting on it goes away. price_level_volume added an order's volume at the price on its created row and subtracted it at the price on whichever later row removed it. Those two prices are not always the same: Bitstamp reports a deleted carrying a price the order never rested at for 1.3% of orders, and the subtraction then landed on a level the volume was never added to, leaving the created level holding it for the rest of the session. Every later row now subtracts at the order's created price, so +v and -v always cancel on one level.

On the bundled Bitstamp sample this removed 104 price levels holding 29.95 BTC that no order was resting on. They were the reported touch on both sides — best bid $78,495.00 against a real best bid of $78,350.00, and best ask $78,324.00 against a real best ask of $78,333.00 — so best_bid_price moves on 35.7% of depth_summary rows and best_ask_vol on 68.9%. The per-order rebuild (engine.book_state) tracks orders by id and never had this problem; the two rebuilds now agree on how long that book is crossed.

aggressiveness_bps is NaN, not an infinity, against a zero touch. The depth engine reports a zero price for an empty side, and the Bitstamp sample also carries orders priced at zero (audit reports these as nonpositive_price). Dividing by that produced a signed infinity that travelled through every downstream mean. A distance from a price that is not tradeable has no value, so it is now NaN.

Changed¶

depth.bin_volume_columns() is public. It returns the per-bps depth-bin volume columns a depth summary carries, ordered from the touch outward, and it was already the answer book_imbalance and depth_signals needed. The feature table needs the same answer, and so does anyone writing a depth feature of their own, so it is no longer private. Behaviour is unchanged.
trade_sign.resolve_direction() now makes the guarantee its docstring already claimed: the direction column it returns holds only "buy" and "sell". A native column was previously passed back untouched however it was filled, and every consumer reads it as == "buy" and takes the rest as a sell — so a partly-labelled feed did not lose its unlabelled trades, it counted them on the wrong side. compute_vpin, order_flow_imbalance, bars and the cost metrics were all affected. Rows that are neither side are now inferred the same way a wholly unlabelled feed is, with a warning saying how many. A feed that labels every trade is passed through unchanged. compute_kyle_lambda reaches the same guarantee: it still requires a direction column rather than inferring one, but a column being present no longer means every row in it is trusted.
trade_sign.prevailing_mid() is now public, and takes allow_exact, skip_crossed, mid_column and require_covered — the last quote strictly before an instant, crossed books skipped, a named reference column such as micro_price, and NaN rather than the final quote reused once the quotes stop reaching. All four default to the previous behaviour, so classify_trade_sign is unchanged. An empty quote frame now returns all NaN instead of raising a pandas MergeError.
Sizes are integer lots plus a lot_size, not floats (issue #226). Breaking: the on-disk schema goes 3.0 → 4.0. Every volume and fill column is now a whole number of lots (int64) instead of a double in the base asset. The base-asset size is lots * lot_size, where lot_size is the instrument's minimum size increment (PipelineConfig.lot_size, default 1e-8; LOBSTER sets 1, whole shares). This is the size half of the integer-tick decision (issue #155) and it fixes a real defect rather than only re-expressing the data.

A price level is a running sum of adds, cancels and fills. A float sum does not return to exactly zero when the last order leaves, so a level landed on residue such as 5.55e-17, stayed live, and was reported as the best bid or ask ahead of the real one. On the bundled Bitstamp sample that corrupted the reported best bid on 25,611 of 313,565 rows (8.2%) and the best ask on 30,096 (9.6%) — the spread on about one row in eleven. Integer lots cancel exactly, so a level empties or it does not, and those counts are now zero.

It was found by the new cross-check against hftbacktest (issue #224), and that is what confirms the fix: replaying an exported session through hftbacktest's own L3 reconstruction now agrees with depth_summary on the best bid and ask for every row across five synthetic seeds, and Nautilus' book agrees too. Before the fix the two disagreed on up to 78 rows a seed.

The change reaches every size-valued column — depth_summary's per-bin volumes, placed_vol and filled_vol, the book snapshot's liquidity, and the queue's ahead_volume and remaining — so their sums are exact as well. Three float-era workarounds went with it: the Kahan compensation behind filled_vol, the simulator's _vol_eps exhaustion tolerance, and the LOBSTER book replay's 1e-12 level cutoff. Loaders convert on the way in; the plots and the round-trip and export writers convert back, so what a user sees and what another tool reads are unchanged. lot_size travels in each Parquet file's key-value metadata under ob_analytics_lot_size, next to ob_analytics_tick_size, and load_data surfaces it as df.attrs["lot_size"]. Files written at 1.0–3.0 still read, as the float-size frames they are. Golden outputs were re-baselined on purpose.

The export writers leave out orders that never rested (issue #224). A marketable order is recorded as a transient add on its own side at the touch, then the fill, then a delete; ob_analytics.depth has always excluded these from the book, but the hftbacktest and Nautilus writers were sending them. A backtesting engine reads an add as real liquidity, so its book crossed at the touch and dropped the resting level the order traded against — its reconstruction drifted permanently thinner than ours. Both writers now exclude them, which is what makes the two books agree.
The order-book engine is its own module (issue #136). The rebuild (order_book), the per-order lifecycles, and the FIFO queue reconstruction moved out of analytics.py / queue.py into ob_analytics/engine/, behind one input and one output: order events in, book states and order lifecycles out, and nothing else. The engine imports no pandas — everything crosses its interface as NumPy arrays, with the shared schema (issue #112) as the input, timestamps as int64 UTC nanoseconds (issue #154) and prices as integer ticks (issue #155). Results carry a row index back into the caller's event arrays instead of copying columns out, so adding a column to the schema does not widen the interface and the engine never learns a vocabulary — order types, venue names — belonging to the layer above. ob_analytics/_engine_frames.py is the one place pandas and the engine meet; analytics.order_book, analytics.order_lifecycles, and the ob_analytics.queue functions are now its frame adapters and keep their exact signatures, dtypes, column order, and index behaviour. Output is unchanged byte for byte — the golden-output gates from issue #143 pass on their recorded fingerprints. Two things did move: the display window (max_levels, bps_range) and the queue sampling window are set by the frame adapters rather than the engine, which reconstructs the whole book and replays to the instants it is given. A new import test (tests/test_engine_boundary.py) keeps the engine free of pandas and of every layer above it. This is what lets the inside be replaced with a faster implementation (#138) or fed one event at a time (#139) without touching anything else. Direction, Action, and Outcome are IntEnum code vocabularies that derive their schema strings from their own member names, so a code and its label cannot drift apart. Two details of the frame code are reproduced deliberately rather than rewritten: an order's executed total is accumulated with compensated (Kahan) summation, as the pandas aggregation it replaced did, and placement values are taken per column as the first non-null among an order's created rows. The lifecycle table is now covered by tests/test_golden_synth.py, which it was not before.
BitstampTradeReader no longer requires integer order ids. It keyed its maker/taker lookup on int(order_id), which crashed on a public trade tape carrying no ids (int(NaN)) and on venues publishing UUIDs. Integer ids behave exactly as before; other ids match on their string form, and a missing id resolves to NaN instead of raising.
One Source shape for every data source, file or live (issue #137; settled #145 as "optional extras plus entry-point plug-ins"). File loaders and live capturers were two separate designs with two registries; they are now one Source protocol with two capability refinements — OfflineSource (replay stored files: the loader / trade-source / writer / depth factories) and LiveSource (capture a venue: snapshot / stream / shutdown_synthetic_events). A source states its level (L2/L3) and feed_type, carries typed settings, and registers in the single SOURCES registry via register_source. A source can be both: BitstampSource now covers offline replay and live capture in one descriptor. This is a breaking API change with no back-compat shims:
- Format → OfflineSource; LiveCapturer → LiveSource; BitstampFormat / LobsterFormat / DepthCsvFormat → BitstampSource / LobsterSource / DepthCsvSource; CcxtCapturer → CcxtSource.
- Pipeline(format=...) → Pipeline(source=...); Pipeline.from_format → Pipeline.from_source.
- The FORMATS / CAPTURERS registries and their register_format / register_capturer / list_formats / list_capturers / get_capturer helpers are replaced by SOURCES / register_source / list_sources / get_source (in ob_analytics.sources).
- PipelineResult.resolution → PipelineResult.level (one coordinate name across the codebase; Source.level, matching the visualization layer).
- CaptureConfig.extras (the untyped settings dict) is removed. Per-source settings are now typed SourceSettings on the source itself, e.g. CcxtSource(settings=CcxtSettings(exchange="binance", depth_limit=100)).
- CLI: process / validate take --source (was --format), and the formats verb is now sources (it also shows each source's capability and required context).
Third-party sources load through entry points. A source can ship in its own package and advertise itself under the ob_analytics.sources entry-point group; ob_analytics.sources.load_source_plugins() discovers and registers it at import time, with no edit to ob-analytics. The built-in sources (bitstamp, lobster, depth_csv, ccxt) self-register on import and stay behind today's [live] / [ccxt] extras.
Prices are now integer ticks, not floats (issue #155). Every price column — events, trades, depth, depth_summary, book snapshot, and order lifecycles — is a whole number of ticks (int64); the quote-currency price is ticks * tick_size, where tick_size is the instrument's minimum price increment (PipelineConfig.tick_size, default 0.01). Loaders convert a raw price to ticks on load; the plots and the round-trip writers convert back for display, so figures and CSV output are unchanged. Storing the exact integer removes the float rounding that made small-tick and 0-1 instruments show crossed levels that were not real, and the depth engine now bins and compares levels on exact integers instead of multiplying and rounding each event — LOBSTER's price_divisor is now just the raw-feed encoding scale. tick_size is written to each Parquet file's ob_analytics_tick_size key-value metadata (a JSON map keyed by instrument, ready for per-(venue, symbol) ticks in #147) and surfaced on load_data frames' attrs. Breaking: the dtype of every price column changed from double to int64 and the stored numbers changed (prices re-expressed as ticks; price-valued analytics such as trade_impacts VWAP and Kyle's λ are now in tick units — multiply by tick_size for the quote currency; scale-free metrics such as bps depth and order-book imbalance are unchanged). The canonical Parquet schema version is now 3.0 (a 1.0 / 2.0 file still reads — Parquet is self-describing — as the float-price frame it stored, whose prices are not directly comparable to a 3.0 file's ticks; re-save it to move it onto the tick model). Golden-output baselines were re-recorded behind the correctness gate (#143).
Timestamps are now tz-aware UTC nanoseconds (timestamp[ns, tz=UTC]) on both clocks — timestamp (receive) and exchange_timestamp (matching engine) — across every table, loader, the synthetic generator, and the toy datasets (issue #154). Before, they were tz-naive and in each venue's native clock (millisecond-resolution UTC for Bitstamp, US/Eastern for LOBSTER), and frames from different venues were declared not comparable. Now every frame sits on one UTC clock, so cross-venue frames can be joined or concatenated directly. LOBSTER's seconds-after-midnight are converted to UTC from the session date and a venue time zone (RunContext(session_tz=...), default America/New_York); Bitstamp / CCXT keep their wall-clock instants and only gain the zone and the nanosecond unit, so their values do not move. The schema also documents a same-instant total order — timestamp, then sequence, then event_id, then ingest_seq (ob_analytics.schemas.time_order_keys), which the per-order reconstructions sort by. Breaking: the dtype of every timestamp column changed, so the canonical Parquet schema version is now 2.0 (a 1.0 file still reads — Parquet is self-describing — as the tz-naive frame it stored; re-save it to move it onto the UTC clock). Consumers that compared pipeline timestamps against tz-naive pandas.Timestamps must now use tz-aware (UTC) ones.

Fixed¶

Order lifecycles read every filled order as cancelled when sizes were floats (#226 regression). order_lifecycles summed each order's fills and cast the total to int64. On integer lots that is exact, but the function also accepts base-asset floats, and it is handed them on every gallery run: display_result converts a whole result to display units before any face builds. Base-asset sizes are mostly below 1, so a 0.121 BTC fill truncated to 0, the order read as never executed, and the three lifecycle-derived L3 faces — Order Activity, Order Outcome and Queue Position — drew a book of nothing but cancellations. On the bundled Bitstamp sample the Order Activity face lost 224 of its 226 filled spans. The sum now keeps the units it was given, integer lots summing exactly and base-asset floats with the compensation that was dropped as part of #226.

LOBSTER was never affected: its lot size is 1, so a truncated size equals the size. Every LOBSTER face is pixel-identical across the change.

LOBSTER's fill column was float64, not integer lots (#226). A 0.0 literal in the expression that built it widened the whole column, so a schema-4.0 LOBSTER run wrote base-asset-looking floats that were really lot counts. Nothing raised; the values only differ from the correct ones once the lot size is not 1.

[0.1.0] - 2026-06-26¶

First public release (PyPI). The Python port of the R obAnalytics package, reworked into a pipeline API with pluggable formats, flow-toxicity metrics, L2/L3 visualization, and Matplotlib/Plotly backends — plus the packaging, documentation, and distribution that make it installable. The sections below also record how the API was deliberately de-bloated and unified during the port (the pipeline's numeric output is unchanged — the regression fingerprints pass; only the shape of the public API moved). See Extending ob-analytics.

Packaging & distribution¶

The bundled Bitstamp sample ships gzip-compressed (orders.csv.gz, ~23 MB → ~2.9 MB installed); sample_csv_path() returns the .gz path, read transparently by pandas. No API change.
Published documentation site (GitHub Pages), CITATION.cff, an explicit GPL-2.0-or-later license section, and a "Scale envelope" doc.
PyPI release workflow (release.yml, trusted publishing), package classifiers and project URLs, and ob_analytics.__version__.
Fixed quickstart and API-reference documentation drift.

Breaking¶

Pydantic models removed. ob_analytics.models (OrderEvent, Trade, DepthLevel, OrderBookSnapshot) deleted; the data contract is now column-list constants + validate_events_df / validate_trades_df / validate_depth_df in ob_analytics.schemas.
metrics/ package removed. ToxicityMetric, Vpin, Ofi, KyleLambda, register_metric, and list_metrics are gone. Call compute_vpin, compute_kyle_lambda, and order_flow_imbalance on result.trades directly.
Pipeline(metrics=...) removed. Metrics are no longer a pipeline stage — compute them after the run.
PipelineConfig.vpin_bucket_volume removed — pass bucket_volume= to compute_vpin.
PipelineResult slimmed to exactly events, trades, depth, depth_summary, and config. The vpin, ofi, metrics, metadata, and extras attributes are gone.
The thirteen plot_* wrappers removed → one plot(name, *, backend="matplotlib", ax=None, **data) dispatcher keyed by (plot_name, backend); renderers self-register into RENDERERS.
Global theme state removed. set_plot_theme / get_plot_theme / _current_theme deleted; pass theme=PlotTheme(...) to plot().
Exception hierarchy collapsed to ObAnalyticsError + ConfigError. InvalidDataError, MatchingError, InsufficientDataError, and ConfigurationError are removed.
Top-level __all__ trimmed to ~22 orchestration names. Low-level helpers now import from their submodules — ob_analytics.bitstamp, ob_analytics.lobster, ob_analytics.analytics, ob_analytics.depth, ob_analytics.data, ob_analytics.visualization, ob_analytics.flow_toxicity.
Format is now a typing.Protocol — there is no base class to inherit; any conforming object is recognised structurally.
Low-level helpers no longer re-exported from the package root (e.g. depth_metrics is now from ob_analytics.depth import depth_metrics).
RunContext.extras and Format.collect_extras removed. LOBSTER trading halts are read from LobsterLoader.trading_halts and composed into the gallery via extra_panels=.
DepthMetricsEngine.update() removed → the public hot-path method is update_side(price, volume, side, out).

Added¶

ob_analytics.schemas — the single data contract: column-list constants (EVENT_COLUMNS, TRADE_COLUMNS, DEPTH_COLUMNS) plus the validate_* functions, run at the pipeline's Protocol boundaries. Replaces the Pydantic model layer.
One generic Registry[K, V] (ob_analytics._registry) backs the format, writer, capturer, and renderer registries. Register through the public helpers register_format, register_writer, register_capturer, and RENDERERS.register / register_plot_backend.
Unified plot() dispatcher + RENDERERS registry keyed by (plot_name, backend), so new plots and backends plug in without a wrapper function. The HTML gallery composes custom panels via extra_panels=.
ob_analytics.live — optional sub-package for live order-book capture: the LiveCapturer protocol (with an optional SupportsDiagnostics capability), CaptureConfig, CaptureResult, CaptureSink, and a generic asyncio runner. Capture output drops straight into the pipeline (orders.csv schema unchanged). Install with pip install "ob-analytics[live]".
ob-analytics capture <venue> CLI verb with a built-in bitstamp capturer (ob_analytics/live/bitstamp.py); --list shows registered capturers. scripts/collect_bitstamp_btcusd.py is now a thin wrapper around it.
TradeSource protocol and BitstampTradeReader — read an authoritative companion trades.csv and join it to events via the fill column.
RunContext dataclass (ob_analytics.protocols, re-exported at the top level) for per-run parameters such as LOBSTER trading_date that don't belong on long-lived Format instances.
Docs — docs/extending.md (add a data source / writer / plot / metric / capturer).
Tests — test_bitstamp.py, test_cli.py (subprocess smoke tests for all CLI subcommands), test_exceptions.py, test_data_registry.py, a regression snapshot suite pinning demo Parquet hashes + the Kyle-λ baseline, and ob_analytics/__main__.py (python -m ob_analytics).

Changed¶

Bundled sample — ob_analytics/_sample_data/ now ships orders.csv and trades.csv from a modern BTC/USD live capture (replaces the legacy 2015 orders-only slice).
Demos consolidated into ob_analytics._demos; scripts/bitstamp_demo.py, scripts/lobster_demo.py, and the bitstamp-demo / lobster-demo CLI subcommands are now thin argparse wrappers. Behaviour unchanged.
Performance — the LOBSTER book is maintained as a SortedDict (no per-event re-sort), Bitstamp trade→event resolution is indexed, LOBSTER depth uses a single strategy, the Plotly import is memoised, and depth metrics sum active levels into bps bins. Numeric output is unchanged (pinned by the regression snapshots).
compute_kyle_lambda computes its OLS via np.linalg.lstsq (was hand-rolled; agrees with the prior implementation to rtol=1e-10).
Internal modules reorganized (renames from the 0.x line): e.g. event_processing.py → bitstamp.py, validation/time helpers → _utils.py, and the visualization modules split into a visualization/ subpackage.
Type checking is Astral's ty (not mypy); lint and format are Ruff.

Removed¶

pacman order type. A legacy artifact of the 2015 Bitstamp HTTP API, where a single order_id could appear at multiple prices over its lifetime. Modern Bitstamp WS v2 and LOBSTER do not produce this pattern (price-modifies become cancel + new id). The type Categorical no longer includes "pacman", the set-subtraction classification path is gone, and LobsterLoader no longer renumbers hidden-execution ids (raw type 5 now retains the native LOBSTER id=0).
Bitstamp trade inference. A companion trades.csv next to orders.csv is now required. Removed: Needleman–Wunsch matching, BitstampMatcher, BitstampTradeInferrer, the MatchingEngine / TradeInferrer protocols, NeedlemanWunschMatcher, and the match_cutoff_ms / price_jump_threshold fields on PipelineConfig.
Zombie detection — get_zombie_ids and the zombie_offset_seconds / skip_zombie_detection config fields.
LOBSTER LobsterMatcher — removed; LobsterTradeInferrer renamed to LobsterTradeReader with load(events, source).
Legacy Bitstamp-only wrappers load_event_data, event_match, match_trades, process_data, and plot_price_levels_faster.
12 unused runtime dependencies (scikit-learn, scipy, jupyter, bokeh, …) and stale dev dependencies (black, flake8 + plugins, darglint).

Fixed¶

depth_metrics no longer overflows for prices > $9,999.99 — dynamic dict[int, int] state replaces the fixed array.
best_bid / best_ask are tracked correctly from the first event (were initialised with dataset-wide max/min).
datetime_to_epoch uses .astype("int64") instead of the deprecated .view("int64").
All print() replaced with loguru logging; all bare assert statements replaced with raised exceptions; plt.show() removed from plot functions (callers control display).