Skip to content

Archive

Trades: 7 days (core and top-100). Core ticker, mark, OI, stats, depth-derived: 10-second samples for 6 months. Core archived as Parquet (backfilling). A backtest wants months of trades, and it reads them end to end rather than at a point in time. The archive is that history as files: one Parquet file per listing, layer and UTC day, written the day after the day is over, checked against the storage it was written to, and listed and fetched through the API.

The principle is the platform’s own: stored as seen. A file holds exactly the rows the platform observed. Nothing is filled, nothing is interpolated, no row is repaired into a smoother series. Where the platform was not listening, the file has no rows, and the gaps of the day are in the list beside it.

group layers in the database in the archive
core every layer below days: see Retention from the day after each day is anchored, with no end
top-100 every layer below days: see Retention from the day after each day is anchored, with no end
tail, new every layer the listing is collected for days: see Retention from the day after each day is anchored, with no end (backfilling from 2026-10-30)

The layers are trade, ticker, mark, depth-derived, open-interest, stats, funding, candle (the 1-minute candles of every price type) and venue-depth. Candle rollups (5 minutes and longer) are computed from the 1-minute candles and are not archived; liquidations and funding settlements stay in the database and are not archived. A day’s row in the database is dropped only after its file has been read back from the storage and its row count and hash are what was written, the database still holds as many rows of that listing and day as the file, and (for every layer but venue-depth) the day’s root is anchored on Base and recorded beside the file. There is no day that is neither in the database nor in the archive because a copy failed.

depth-derived is the book’s derived figures (mid, spread, liquidity in the bands around the mid and the value resting in them, the number of levels), not the book itself: levels are never stored.

Terminal window
curl -s -H "Authorization: Bearer $DEBYKO_KEY" \
"https://api.debyko.com/v2/archive?listing=BINANCE-USDM:BTCUSDT&layer=trade&from=2026-09-28&to=2026-09-30"
{
"listing": "BINANCE-USDM:BTCUSDT", "from": "2026-09-28", "to": "2026-09-30", "truncated": false,
"items": [
{ "layer": "trade", "day": "2026-09-30", "group": "core", "available": true, "rows": 4182113, "bytes": 187004211,
"sha256": "9f2c…e1", "gaps": 1, "anchored": false,
"download": "/v2/archive/BINANCE-USDM%3ABTCUSDT/trade/2026-09-30",
"coverage": [ { "cause": "ws_disconnected", "start": "2026-09-30T03:12:09.114000Z", "end": "2026-09-30T03:12:31.480000Z",
"end_unknown": false, "scope": "segment", "datasets": null, "repaired_at": null } ] }
]
}

(The figures are illustrative.) The list is free with any key and counted against nothing. It shows only files that were read back and checked. A day on which nothing was observed has available: false, no hash and no download, and its coverage says why.

Terminal window
curl -sL -H "Authorization: Bearer $DEBYKO_KEY" -o btcusdt-trade-2026-09-30.parquet \
"https://api.debyko.com/v2/archive/BINANCE-USDM%3ABTCUSDT/trade/2026-09-30"

The answer is a 302 to a link that reads that one file for fifteen minutes. The response carries X-Debyko-Sha256 (the file’s hash) and Link (the day’s entry in the list). The link needs no key and carries none: a client that drops the Authorization header when it follows a redirect to another host, as curl -L and requests do, never sends your key to the storage. The contents of a file are never paged through the API: the redirect is the whole of it.

  • With a key, a file counts against the plan’s files for the calendar month (UTC): Free and Mini none, Standard 100, Pro 1 000, Pro Plus 10 000. Past them the answer is 402 with the price of the file; X-Debyko-Archive-Remaining says what is left.
  • Without a key, when paying per call is on, a file costs 0.10 USDC (core) or 0.05 USDC (top-100): one payment, one file, one 302.
  • A day that is not archived yet is a 404: a day is written once it has been over for two days.

When as_of or a history request reaches a raw day the database no longer holds, the answer is not_retained as ever, and when a file exists for that day it carries archive: { "path": …, "sha256": … } beside the reason. That is a pointer and nothing more; the file is fetched as above.

  • Path in the storage: archive/{venue}/{listing}/{layer}/{yyyy}/{mm}/{listing}-{layer}-{yyyy-mm-dd}.parquet, the venue in lower case, the symbol with every character outside A-Za-z0-9._- written as %XX.
  • Rows: in time order (trades: then by the venue’s trade id), one row per observation, exactly as stored. A file has the rows of one listing and one UTC day.
  • Row groups: one per hour, a very busy hour in groups of at most 25 000 rows, each with minimum and maximum statistics on every column, so a reader can skip the hours it does not need. Compression: zstd.
  • Types: timestamps are INT64 with the logical type timestamp, UTC, nanosecond unit (the platform keeps microseconds; they are written as they are). The manifest records schema_version; v2 writes it in the Parquet footer too. Older v1 footers have no version key and are treated as v1. In v2, venue-published numeric columns (including the mixed-provenance premium) are exact decimal strings, with no float conversion or rounding. Derived-only columns, including mid and spread, remain DECIMAL(38, 18). Existing v1 files stay unchanged: their numeric columns are DECIMAL(38, 18). generation, sequence are INT64, level counts INT32, words are strings, and suspended is BOOLEAN.
  • Columns, in this order, per layer. ? marks a column that can be null (the venue did not publish it).

trade — event_time, price, size, side, venue_uid, received_at, generation, sequence, origin

ticker — bucket, bid, ask, last?, bid_size?, ask_size?, received_at, source_time?, generation, sequence, transport, origin, cadence?

mark — bucket, price, index_price?, premium?, received_at, source_time?, generation, sequence, transport, origin, cadence?

depth-derived — bucket, mid, spread, spread_bps, imbalance?, bid_liquidity_01?, bid_liquidity_025?, bid_liquidity_05?, bid_liquidity_1?, ask_liquidity_01?, ask_liquidity_025?, ask_liquidity_05?, ask_liquidity_1?, bid_value_01?, bid_value_025?, bid_value_05?, bid_value_1?, ask_value_01?, ask_value_025?, ask_value_05?, ask_value_1?, bid_levels, ask_levels, bid_band_levels_01?, bid_band_levels_025?, bid_band_levels_05?, bid_band_levels_1?, ask_band_levels_01?, ask_band_levels_025?, ask_band_levels_05?, ask_band_levels_1?, received_at, source_time?, generation, sequence, transport, origin, cadence?

open-interest — bucket, value, received_at, source_time?, generation, sequence, transport, origin, cadence?

funding — bucket, rate, relative_rate?, predicted_rate?, predicted_relative_rate?, next_funding_at?, short_rate?, predicted_short_rate?, received_at, source_time?, generation, sequence, transport, origin, cadence?

candle — open_time, price_type, open_price, high_price, low_price, close_price, volume?, trade_count?, received_at, source_time?, generation, sequence, transport, origin

venue-depth — bucket, bid_01?, ask_01?, bid_025?, ask_025?, bid_05?, ask_05?, bid_1?, ask_1?, received_at, source_time?, generation, sequence, transport, origin, cadence?

stats — bucket, volume_24h?, quote_volume_24h?, open_24h?, high_24h?, low_24h?, change_percent?, suspended?, received_at, source_time?, generation, sequence, transport, origin, cadence?

Derived columns are the platform’s own calculation, not a venue’s figure, and are listed in the file’s footer (debyko.derived_columns). What the platform computes has the precision of its unit, half to even, fixed where it is computed (from 2026-10-02; before that a division carried up to 28 digits): a fraction or ratio has 12 digits after the decimal point, basis points have 6, and a price or quantity is exact arithmetic, as stored, with no rounding of ours. Where a stored historical value still has more digits than its unit, the file holds it rounded half to even to the digits of the unit (to 18, the column’s scale, for an amount): that is the only rounding there is, and only these columns have it. The mark layer’s premium is the one derived column that is a venue’s own figure on some rows (Hyperliquid, OKX and Kraken Futures state it; the platform divides it for Bybit, Binance, Coinbase and Weex, to 12 digits when it computes it), so v2 holds its stored decimal text exactly. A v1 file, written before this version, holds the value at the old 18-digit scale.

Layer Columns Kind
trade, ticker every number venue figure, never rounded
open-interest, stats, funding, candle, venue-depth every number venue figure, never rounded
mark price, index_price venue figure, never rounded
mark premium derived: fraction, 12 digits when the platform computes it (a stated premium is kept to 18)
depth-derived mid, spread derived: amount, as stored (at most 18 digits); exact arithmetic on venue prices
depth-derived spread_bps derived: basis points, 6 digits
depth-derived imbalance derived: fraction, 12 digits
depth-derived bid_liquidity_01, bid_liquidity_025, bid_liquidity_05, bid_liquidity_1, ask_liquidity_01, ask_liquidity_025, ask_liquidity_05, ask_liquidity_1 derived: quantity, as stored (at most 18 digits)
depth-derived bid_value_01, bid_value_025, bid_value_05, bid_value_1, ask_value_01, ask_value_025, ask_value_05, ask_value_1 derived: amount in the quote currency, as stored (at most 18 digits)

bucket is the start of the sampling window the row stands for (cadence says how long it is: 1s, 10s, 15s or 1m); received_at is when the platform received the value and source_time when the venue says it happened. The same words mean the same as in the snapshots. Not in a file: the instrument (the file is of one listing) and the storage class a row was filed under, which is the platform’s own routing.

A file is not a claim that the market was quiet where it has no rows. It is the rows the platform saw. The gaps of the day (the platform’s connection to the venue broke, a poll was rate limited, the collector was down) are in coverage of the list entry with their causes, copied from the platform’s gap record when the file was written: scope says whether the whole venue connection was affected or this listing alone, datasets which layers, repaired_at whether a replay later filled it (such rows are in the file, with their own origin). A day with a gap is still written, rows as observed. A day with no rows at all has no file, and an entry that says why.

Each takes the downloaded file or, for the first two and DuckDB, the link itself (the Location of the 302; it is good for fifteen minutes).

# pandas
import pandas as pd
import decimal
df = pd.read_parquet("btcusdt-trade-2026-09-30.parquet") # or the https link; v2 prices are strings, v1 prices are decimal.Decimal
df["price"] = df["price"].map(decimal.Decimal) # exact for both versions
# polars
import polars as pl
df = pl.read_parquet("btcusdt-trade-2026-09-30.parquet") # or pl.scan_parquet(link) to read only the row groups a filter needs
df = df.with_columns(pl.col("price").cast(pl.Decimal(38, 30))) # only when this precision and scale fit your values
-- DuckDB
SELECT count(*), min(event_time), max(event_time) FROM read_parquet('btcusdt-trade-2026-09-30.parquet'); -- or the https link
SELECT TRY_CAST(price AS DECIMAL(38, 30)) AS price FROM read_parquet('btcusdt-trade-2026-09-30.parquet'); -- NULL when it does not fit

Decimal casts in Polars and DuckDB require a precision and scale that fit your data; keep the original string for values outside their range.

Every file has a SHA-256 in the list and in the X-Debyko-Sha256 header of the download; what you download hashes to it (sha256sum file.parquet). The hash is what the manifest says the platform wrote and then read back from storage. New files also have a rows_hash: SHA-256 of the canonical row stream (JCS row arrays in schema column order, one per line), with numeric values represented as decimal strings. This hash is independent of the v1/v2 Parquet encoding and is the planned anchor leaf; the file SHA-256 checks the downloaded bytes. Existing v1 files are not rewritten and their manifest rows_hash remains empty until a separate read-only backfill. Anchoring is not live yet: anchored is false in the list.