Skip to content

Hidden liquidity

Size a venue will trade that the visible book does not show. An L3 stream records two footprints of it: the refills of an iceberg order, and trades that print inside the visible spread.

See Find hidden liquidity for a worked example and the measured recall and precision, and Glossary: hidden liquidity for the terms.

Iceberg orders

detect_icebergs 

detect_icebergs(
    events: DataFrame,
    trades: DataFrame,
    *,
    max_delay: str | Timedelta = ICEBERG_MAX_DELAY,
) -> IcebergDetection

Find suspected iceberg orders from the refills they leave behind.

A slice is filled out when its last fill as a maker leaves it with nothing outstanding. A refill is the first new order placed at the same side and price at or after that fill, within max_delay, and, at the same instant, after it in event order. When several slices at one price level are filled out together, a new order is matched to the one whose displayed size it equals, and otherwise to the one filled out first. Each order joins at most one refill as the new slice and at most one as the old.

Parameters:

Name Type Description Default
events DataFrame

L3 events with event_id, id, timestamp, price, volume, action, direction and fill.

 required
trades DataFrame

Trades with maker_event_id. Only a maker's fill can empty a displayed peak, so an aggressor filled to zero is never a slice.

 required
max_delay str or Timedelta

Longest wait from a peak being filled to its refill. See :data:ICEBERG_MAX_DELAY for the default and why.

 ICEBERG_MAX_DELAY

Returns:

Type Description
IcebergDetection

The suspected icebergs and the slices each one is made of. Both frames are empty, with their columns, when none is found.

Raises:

Type Description
ConfigError

If a required column is missing, or max_delay is negative.

ICEBERG_MAX_DELAY module-attribute 

ICEBERG_MAX_DELAY = Timedelta('1ms')

Trades against hidden orders

hidden_trades 

hidden_trades(
    events: DataFrame,
    trades: DataFrame,
    depth_summary: DataFrame,
) -> pd.DataFrame

Return the trades that printed strictly inside the visible spread.

Each trade is compared with the spread standing just before its maker's fill: the depth summary is read by an as-of join at the maker event's timestamp, with that instant itself excluded. The maker event is used rather than the trade's own timestamp because some feeds report the fill on the order stream before the trade print arrives, and by the print the maker has already left the book. Excluding the instant stops a sweep that empties a price level from making its own later prints look inside the spread. A trade with no maker event falls back to its own timestamp.

A trade is kept when both sides of the book were present and not crossed, and best_bid_price < price < best_ask_price. The depth summary holds visible orders only, so such a trade executed against an order the book did not show. A hidden order resting at or behind the touch is missed: its trades print at a visible price.

Parameters:

Name Type Description Default
events DataFrame

L3 events with event_id and timestamp.

 required
trades DataFrame

Trades with timestamp, price and maker_event_id.

 required
depth_summary DataFrame

The run's depth summary, with timestamp, best_bid_price and best_ask_price.

 required

Returns:

Type Description
DataFrame

The matching rows of trades, index kept, with the standing best_bid_price and best_ask_price added, in depth_summary's own dtype (integer ticks on the canonical schema).

Raises:

Type Description
ConfigError

If a required column is missing.

Models

IcebergDetection dataclass 

IcebergDetection(icebergs: DataFrame, slices: DataFrame)

Result of :func:detect_icebergs.

Attributes:

Name Type Description
icebergs DataFrame

One row per suspected iceberg, in order of start:

  • iceberg — number of the iceberg, from 1.
  • direction, price — the side and price level it rested at.
  • start — when the first slice was placed (or first seen).
  • end — the last event of the last slice.
  • slices — visible orders in the chain, the first one included.
  • refills — slices - 1.
  • same_size_refills — refills whose displayed size equals the displayed size of the slice they replaced.
  • peak — displayed size of the first slice, in lots; <NA> when the first slice was already resting when the stream began.
  • executed — size filled across all slices, in lots.
  • median_delay_s — median wait from a peak filled to its refill.
  • confidence — high when there are at least two refills and every one matches the displayed size, medium when at least one does, low otherwise. An ordered categorical.
slices DataFrame

One row per visible order in a chain: iceberg, slice (from 1), order id, timestamp placed (or first seen), displayed volume, total fill, and delay_s from the previous slice being filled to this one appearing (NaN for the first slice).