Market Lab v0.0.5

Market Lab moves beyond analysis into a complete execution runtime. This release adds standalone Binance Spot and USD-M historical data, BULK testnet and Hyperliquid mainnet or testnet trading, persistent live jobs, a redesigned scripting contract, faster execution-aware backtests, built-in TWAP, VWAP, and OIWAP execution strategies, and native market-making bots.

Binance Integration

Standalone Binance provider implemented by Kampouse.

• Added Binance as a standalone, unauthenticated market-data provider.
• Added binance for Binance Spot and binancef for Binance USD-M perpetual futures.
• Added refreshable binance-markets.json and binancef-markets.json snapshots through mlab markets --exchange <exchange> --refresh.
• Added public REST health checks through mlab health --exchange binance and mlab health --exchange binancef.
• Added historical candle requests through source candles.
• Added historical volume bars derived from candle volume through source volumes.
• Added standalone script-backtest selectors: candles@binance, candles@binancef, volumes@binance, and volumes@binancef.
• Added automatic historical pagination with a 5,000-record safety limit.
• Kept standalone Binance separate from MMT. Direct Binance uses --exchange; MMT Binance access uses --provider mmt --exchange <exchange>.
• Binance execution, live WebSocket streams, orderbooks, open interest, funding, and statistics are not included.

BULK Integration

• Added BULK as a standalone exchange for public market data and live execution.
• Standardized BULK perpetual market data, script sources, and execution on the bulkf identifier. The bare bulk market name is reserved for future spot support, while BULK authentication remains mlab auth set bulk.
• BULK execution is testnet-only because BULK does not yet provide a public mainnet. It continues to use --venue bulkf without a --testnet flag.
• Added a versioned BULK market catalog with symbol mappings, tick sizes, lot sizes, minimum notionals, supported order types, time-in-force rules, and maximum leverage.
• Added normalized BULK candles, orderbooks, trades, volume delta, open interest, volume bars, funding, ticker statistics, and market metadata.
• Added BULK HTTP and WebSocket clients with normalized timestamps and account-aware data models.
• Added refreshable Binance Spot, Binance USD-M futures, BULK, Hyperliquid, and MMT market snapshots stored under ~/.market-lab/markets.
• Added explicit spot or futures classification to every exchange snapshot, with constant-time runtime lookup and local rejection of spot open-interest requests.
• Added mlab markets --exchange bulkf --refresh and MMT provider refresh for explicitly updating the installed snapshots.
• Refreshing BULK markets now installs bulkf-markets.json and removes the retired bulk-markets.json snapshot.
• Added mlab markets --exchange bulkf for offline lookup of installed execution rules without a credential.
• Standardized provider routing: standalone exchanges use --exchange binance, --exchange binancef, --exchange bulkf, or --exchange hyperliquidf; multi-exchange MMT sources use --provider mmt --exchange <exchange>.
• Added BULK source support to scripts through selectors such as candles@bulkf:timeframe=5 and orderbook@bulkf:depth=20.

Hyperliquid Integration

• Reserved hyperliquid for future spot support. It now returns a clear unavailable-spot error.
• Made hyperliquidf the canonical name for Hyperliquid core perpetuals across standalone data, MMT data, execution, scripts, strategies, bots, jobs, and source selectors.
• Made hyperliquidf-xyz the canonical MMT name for upstream hyperliquid-xyz TradeXYZ HIP-3 markets.
• Kept Hyperliquid authentication under mlab auth set hyperliquid; the naming migration changes exchange and venue selectors, not the auth key.
• Stored both the canonical Market Lab name and upstream MMT name in snapshots through exchange and providerExchange.
• Preserved compatibility with jobs and snapshots created before the naming migration.
• Added Hyperliquid as a standalone public-data exchange and execution venue through --exchange hyperliquidf and --venue hyperliquidf.
• Added native-perpetual execution on Hyperliquid mainnet and testnet. Mainnet is the default; --testnet selects testnet. Spot, HIP-3 markets, vaults, and subaccounts are not included.
• Added --testnet across Hyperliquid direct trades, account queries, cancel and close commands, scripts, strategies, and bots.
• Added a refreshable mainnet hyperliquidf-markets.json catalog containing normalized symbols, futures classification, native coin mapping, size precision, lot size, leverage limits, and execution capabilities. Testnet market metadata is resolved at execution time.
• Added offline Hyperliquid market lookup and local market validation through mlab markets --exchange hyperliquidf.
• Added public Hyperliquid health, historical and live candles, current and live orderbooks, current and live OI, historical and live volume bars, trade-derived live VD, one-day statistics, funding, and normalized trades.
• Added Hyperliquid WebSocket streams for orderbooks, trades, candles, asset context, order updates, fills, and user/account events.
• Added direct scripting selectors such as candles@hyperliquidf, orderbook@hyperliquidf, oi@hyperliquidf, volumes@hyperliquidf, and vd@hyperliquidf.
• Added direct Hyperliquid candle and volume backtests at venue-native intervals; historical orderbooks, VD, and direct OI remain unavailable.
• Added arbitrary second-based live Hyperliquid script candles built from trades with startup partial-bucket omission.
• Added Hyperliquid API-agent authorization through mlab auth set hyperliquid. One hidden master-key prompt approves separate mainnet and testnet agents for the same account; the master private key is never stored.
• Stored only the main address and both network-specific agent credentials in the owner-only local credential store.
• Added mlab auth set hyperliquid --reauthorize to replace both network-specific agents.
• Added Hyperliquid market and limit execution, GTC/IOC/ALO time-in-force, reduce-only orders, cancellation, full position closing, and whole-number cross-leverage updates.
• Implemented Hyperliquid market requests as IOC limits with a 50 bps reference-price guard rather than an unbounded market wire order.
• Added native Hyperliquid stop-loss and take-profit triggers through normalTpsl grouping without claiming OCO sibling cancellation.
• Added normalized Hyperliquid positions, open orders, fills, liquidation values, funding, and account lifecycle events using the main account for queries and the API wallet for signatures.
• Sent signed Hyperliquid actions through the HTTP exchange endpoint while public and private lifecycle state remains WebSocket-driven inside mlabd.
• Extended CLI trading, scripts, TWAP, VWAP, OIWAP, Mid-Price, and Volume-Mid to execute through the selected BULK or Hyperliquid venue adapter.
• Extended VWAP's direct trajectory sources, execution-volume feed, orderbook guard, fills, and participation capacity to Hyperliquid.
• Kept OIWAP trajectory inputs MMT-only while allowing Hyperliquid execution; direct execution-venue OI is not mixed into an MMT historical trajectory.

Authentication and Account Security

• Added BULK agent-wallet authorization through mlab auth set bulk.
• The main wallet private key is accepted only through the hidden authorization prompt and is never persisted.
• Generated agent credentials are stored in owner-only files under ~/.market-lab/credentials.
• Removed the desktop keychain dependency so authentication works on headless Linux and unattended daemon restarts do not require D-Bus, GNOME Keyring, or a macOS password prompt.
• Added mlab auth set bulk --reauthorize to repair or replace an invalid delegated signer without deleting local state manually.
• Added signer/account validation before every live mutation.
• Added actionable unauthorized-signer errors that point directly to the reauthorization command.

Trading and Execution

• Added direct perpetual commands for opening long and short positions.
• Added market and limit orders, GTC/IOC/ALO time-in-force, reduce-only execution, cancellation, and position closing.
• Added --margin as the collateral-based sizing input. Exposure is calculated as margin multiplied by leverage.
• Retained --size for exact base-asset exposure without leverage multiplication.
• Added leverage validation against the selected market and venue-specific leverage updates before applicable orders.
• Added native stop-loss and take-profit protection through BULK on-fill conditional orders.
• Added dry-run trade plans, interactive confirmation, normalized order receipts, and structured JSON output.
• Added account commands for positions, open orders, fills, cancellation, and position closing.
• Added estimated margin and exposure to trade plans and reports.
• Added projected liquidation-price output support; the value remains unavailable when BULK cannot provide a pre-trade portfolio simulation.
• Added deterministic BULK order-ID retention before submission.
• Moved signed order, cancellation, leverage-update, and agent-authorization submission from HTTP to BULK's trading WebSocket. Daemon execution reuses a dedicated persistent connection; HTTP remains the read and reconciliation path for account snapshots and uncertain submission outcomes.
• Accepted both nested and flat BULK WebSocket action-response envelopes and surfaced uncorrelated protocol errors instead of misreporting a missing payload.
• Treated BULK post acknowledgements as transport receipts instead of venue outcomes. Leverage acknowledgements now allow the order submission to proceed, acknowledged orders are tracked optimistically by deterministic order ID, and the account stream remains authoritative for resting, fill, rejection, and cancellation states.
• Added ambiguous-timeout reconciliation against orderHistory, openOrders, and fills so an accepted order is not reported as failed or submitted twice.
• Reconciled executions retain the original order ID and are identified in strategy logs.
• Explicit exchange rejections remain failures and are never converted into successful reconciliations.

Persistent Runtime

• Added the mlabd execution daemon as a separate packaged binary.
• Added automatic daemon startup and runtime-version negotiation from the CLI.
• Added owner-only local IPC, persisted runtime state, execution journals, and runtime error reporting.
• Added mlab daemon start, status, stop, and events commands.
• Moved live account, order, fill, and position updates from polling to a dedicated BULK account WebSocket, separate from the trading WebSocket.
• Added reconnect handling, gap recovery, account-stream health, and tracked-order lifecycle updates.
• Added detached script and strategy worker processes so the terminal is released after deployment.
• Added job status, heartbeat, stop, failure, completion, and log-file management.
• Added safe decoding of persisted jobs created before newly added TWAP margin fields.
• Made daemon replacement tolerate the expected empty connection during shutdown, preventing a protocol upgrade from failing midway through the stop-and-start handover.

Scripting Runtime

• Reworked scripts around one event-driven contract shared by live runs and backtests.
• Added live trade sources through qualified selectors such as trades@bulkf, trades@hyperliquidf, and trades@binancef@mmt.
• Added normalized script trade records containing only price and size; symbol, exchange, provider, timestamp, and aggressor side are not duplicated in the record.
• Live scripts now receive one onData event for every trade. Trade sources are rejected in backtests with a clear live-only error.
• Reused one MMT trade subscription when a script requests both raw trades and trade-derived candles for the same market.
• Removed script mode and manifest clock; every configured source event can now drive onData.
• Added detached live script jobs with script jobs, status, logs, and stop commands.
• Added optional script duration. Omitted duration runs indefinitely.
• Added ctx.trade(), ctx.order(), and ctx.cancel() for live and simulated execution.
• Added the asynchronous onExecution hook for live and simulated order lifecycle events.
• Added deterministic script order keys and idempotent command handling.
• Added input.positions.open so scripts can inspect current managed positions.
• Added native stop-loss and take-profit fields to scripted entry orders.
• Added market and limit order support with time-in-force validation.
• Added explicit perpetual position actions: open-long, open-short, close-long, and close-short.
• Added raw buy/sell order placement through ctx.order; long aliases buy and short aliases sell.
• Raw non-reduce-only orders follow one-way venue netting and may increase, reduce, close, or flip inventory. Raw reduce-only orders never increase or flip it.
• Backtests deliver simulated pending, accepted, fill, filled, and cancelled events through onExecution, allowing the same order-replacement logic in both environments.
• Kept partially filled script orders active until their actual terminal update instead of treating the first partial fill as completion.
• Added daemon-owned managed-order cleanup when a script is stopped, reaches its duration, or fails, so resting quotes are not orphaned after the worker exits.
• Added supervised live-source reconnection with one-to-30-second exponential backoff while worker heartbeats and execution-event delivery continue.
• A source disconnect now cancels the job's non-terminal managed orders before reconnecting, preventing stale quotes from remaining exposed without market data.
• Added source disconnect/reconnect messages and the final terminal worker error to script logs, so failed jobs no longer require inspection of worker.log.
• Distinguished daemon receipt time from venue event time in the scripting event contract; BULK retains its consensus-clock timestamp inside the venue payload.
• Close actions are reduce-only internally and default to closing the full matching position when size is omitted.
• Opposite directional entries require an explicit close first instead of silently reversing the position.
• Added live execution gating through --venue bulkf or --venue hyperliquidf; market-data access alone never enables trading.
• Removed returned signal/intent execution. Scripts perform actions directly and do not need to return a value.
• Reduced live script logs to meaningful lifecycle, output, and execution events instead of appending the full hook payload continuously.
• Added compact human-readable script logs output while retaining the complete event stream through --output jsonl.
• Preserved BULK terminal rejection names such as rejectedCrossing when the venue returns only an order ID in the rejection details.
• Updated the mid-price maker example to replenish only from a fresh orderbook event, avoiding immediate post-only resubmission at a stale filled price.
• Moved the maker demo one real book level behind the touch to reduce post-only crossing races while keeping every quote aligned to the venue tick.
• Deduplicated repeated terminal order events in human-readable script logs.
• Made managed-order terminal states irreversible so delayed snapshots, recovery rows, or account-stream updates cannot move a filled, cancelled, or rejected order back to resting.

Multi-Exchange Sources

• Added exchange- and provider-qualified selectors for scripts.
• MMT selectors support forms such as candles@binancef@mmt:timeframe=60 and orderbook@lighterf@mmt:depth=20.
• Standalone exchanges remain selectors, for example candles@binancef:timeframe=60, orderbook@bulkf:depth=100, and orderbook@hyperliquidf:depth=20, even when a script also consumes MMT sources.
• A single script can consume the same source type from multiple exchanges and providers simultaneously.
• Script manifests declare required source kinds while runtime selectors determine the concrete exchanges, providers, and source parameters.
• Added selector-keyed access through history.source(selector).
• Removed redundant nested shapes such as input.candles.candles.
• A selector requested by a script but not configured at runtime remains empty rather than silently binding to another exchange.
• Added source metadata describing the triggering selector, exchange, provider, timeframe, bucket, and depth.
• Added TOML source arrays and CLI-over-config merging for qualified selectors.
• Script parameters can be declared as direct TOML keys without an unnecessary grouping table.

Live History

• Added bounded per-selector history to the third onData(ctx, input, history) argument.
• Added history.source(selector) for the retained list and history.source(selector, offset) for one record.
• Offset 0 is the newest record, offset 1 is the previous record, and unavailable offsets return no record.
• History is isolated per exact selector, so multiple exchanges never overwrite one another.
• Added configurable manifest lookback retention with a minimum of two records and a maximum of 5,000.
• Repeated updates with the same timestamp replace the current record instead of creating false historical bars.
• Live history begins when the script starts and does not imply provider-side historical backfill.

Live Time-Series Aggregation

• Added trade-stream aggregation for live MMT, BULK, and Hyperliquid candle construction.
• Live scripts can request custom second-based candle timeframes instead of waiting for exchange-native intervals.
• The incomplete interval in progress when a script starts is discarded.
• Emission begins at the next clean timeframe boundary to avoid publishing partial OHLCV data.
• Historical backtests remain limited by the granularity stored by the selected provider.

Script Studies and Runtime Safety

• Added persistent QuickJS sessions instead of rebuilding the JavaScript runtime for every source event.
• Added bounded runtime resources, cancellation, hook telemetry, crash reports, and script run history.
• Exposed Rust study implementations directly to JavaScript.
• Added SMA, EMA, CVD, spread, depth, imbalance, slippage, and VAMP helpers under ctx.study.
• Added input validation and stable normalized return shapes for study helpers.
• Added incremental source history so moving studies naturally operate from script start through the current event.

Backtesting

• Reworked script backtests around one chronologically merged timeline containing every configured source event.
• Removed candle-only clocking; orderbook and other source updates can now trigger script logic during a backtest.
• Added a persistent JavaScript session and incremental history, substantially reducing multi-thousand-record execution time.
• Added simulation for market orders, resting limit orders, cancellation, and next-event limit fills.
• Added native stop-loss, take-profit, and deterministic same-bar protective-order handling.
• Added explicit open/close position simulation matching the live perpetual action contract.
• Added partial and full close support, reduce-only semantics, and prevention of accidental directional reversal.
• Added net managed-position behavior instead of creating an independent open position for every signal.
• Added realized and unrealized PnL, capital required, return, win rate, profit factor, average/best/worst trade, Sharpe ratio, and maximum drawdown.
• Added compact trade and open-position output with --verbose for the full ledger.
• Added analysis-only backtests; a script is no longer required to call ctx.trade() merely to complete.
• Added multi-exchange backtests using the same exact selectors as live scripts.

Built-In Strategies

• Removed the legacy Rust SMA-crossover strategy and its old command structure.
• Added TWAP as the first built-in execution strategy.
• Added side, margin or exact size, leverage, duration, child interval, reduce-only, dry-run, and confirmation inputs.
• TWAP divides the normalized parent size across the full schedule without dropping lot-size remainders.
• Live TWAP runs as a detached mlabd job so the initiating terminal remains free.
• Added TWAP plan output, per-child logs, job status, log following, stopping, completion summaries, and failure reports.
• Added strict child sequence and plan validation inside the daemon.
• Every child order carries the configured leverage and sizing semantics.
• Stopping a TWAP prevents future children; it does not reverse children already filled.
• Timed-out child submissions are reconciled by deterministic order ID before the schedule continues.
• Added VWAP with consolidated one-minute volume trajectories from direct exchanges and MMT exchange selectors.
• Added maker-first ALO execution, job-owned cancel/replace, live fill reconciliation, dynamic taker catch-up, deadline handling, and a 20 bps depth guard.
• VWAP accepts duration and optional volume sources but intentionally exposes no child interval.
• Added degraded market-data handling that substitutes forecast progress when a live volume source disconnects.
• Separated aggregated trajectory volume from execution-venue capacity so high-volume reference exchanges cannot force oversized orders onto a thinner execution venue.
• Added always-on execution-venue volume tracking, cumulative participation credit, working-order reservation, a 10% participation cap, and preflight shortfall detection.
• Switched live VWAP volume accumulation to normalized exchange trade streams; historical one-minute volume remains responsible only for forecast-curve construction.
• Added OIWAP with multi-exchange open-interest activity trajectories and the same supervised maker-first execution model.
• Added required exchange@mmt OI source selectors with installed-market validation, duplicate rejection, and futures/perpetual-only enforcement.
• OIWAP constructs its forecast from 28 days of normalized one-minute MMT OI history using each venue's absolute open-to-close change before aggregation, preventing opposing venue changes from cancelling one another.
• Added live MMT OI trajectory tracking from the same sources used by the forecast, including clean startup-minute omission and forming-candle revision without double-counting.
• starts all selected OI-source history requests concurrently to avoid serial startup delays on multi-exchange 28-day curves.
• Kept execution capacity independent from OI activity: execution-venue trades create participation credit, while orderbook depth, fills, the 10% cap, and the 20 bps taker guard constrain every child.
• Deliberately excluded direct execution-venue OI from live OIWAP progress because it is not part of the normalized MMT historical trajectory.
• Added OIWAP feasibility plans, detached jobs, degraded-source handling, structured logs, status reporting, cancellation, and completion summaries.
• Added a non-blocking OIWAP side advisory from 15 minutes of signed multi-exchange OI change and execution-venue price movement, including regime, directional bias, source agreement, confidence, and requested-side alignment.
• Countertrend OIWAP sides now receive a visible warning without being changed or rejected; incomplete advisory data never blocks execution.

Native Bots and Market Making

• Added a dedicated mlab bot command group for autonomous native Rust bots, separate from user-directed execution strategies and JavaScript scripts.
• Added the first built-in bot, mid-price, with detached daemon jobs, dry-run plans, confirmation, status, compact logs, stopping, and persisted lifecycle state.
• Added configurable duration, a non-negative midpoint spread, leverage, and exact-size or margin-based maximum-inventory sizing.
• Added an authoritative local orderbook built from the initial BULK snapshot and subsequent price-level deltas. Positive quantities insert or update a level, zero quantities remove it, and reconnects discard stale state until a new snapshot arrives.
• Retained the complete known book in ordered BTreeMap sides so deleting the best level immediately reveals the correct next level without a REST request, full sort, or depth-copy operation.
• Added a zero-allocation top-of-book path for native strategies and bots. Full depth vectors are created only when an external consumer explicitly requests a snapshot.
• Removed the second bid/ask allocation from typed BULK WebSocket orderbook decoding by applying decoded levels directly to the local book.
• Separated the market-data and execution paths: public orderbook updates drive midpoint quotes, while the private account stream supplies authoritative fills and order states.
• Added lossless fill delivery with reconnect reconciliation and occurrence-aware deduplication, including multiple otherwise identical partial fills in the same normalized timestamp.
• Added coalesced book delivery so a slow venue action cannot create a growing queue of obsolete market states; the controller always evaluates the newest available touch.
• Added independent asynchronous bid and ask submit/cancel state, post-only ALO enforcement, tick-aligned midpoint pricing, and rapid retry from a fresh book after a crossing rejection.
• Made margin × leverage the hard one-sided inventory exposure, with half quoted on each side while flat and continuous replenishment until the configured duration ends.
• Added linear inventory skew that shrinks the risk-increasing quote and grows the inventory-reducing quote, with hard headroom caps at the long and short inventory limits.
• Added percentage-based directional bias from -100 to 100 as a size tilt combined with the inventory skew; it does not change midpoint pricing or silently increase the inventory limit.
• Split midpoint market making into two execution modes. mid-price protects the configured spread with a 500ms minimum quote age and symmetric replacement after the larger of two ticks or 0.25 bps; volume-mid prioritizes fills with explicit --refresh-time and --refresh-tolerance-bps controls and only chases the market when a quote moves away.
• Kept fills, inventory-headroom changes, market-data loss, completed order actions, and crossing retries independent of the normal price-refresh window so risk and replenishment reactions remain immediate.
• Filtered transient crossed touches caused by consecutive bid/ask deltas before they reach the execution controller, and replaced the crossing-retry boolean with monotonic book-revision gating so a quote side cannot remain paused after the market advances.
• Preserved rejectedCrossing in native bot logs instead of reducing it to an unexplained generic rejection.
• Kept Mid Mode pricing strictly attached to the current orderbook midpoint; inventory entry price never anchors replacement quotes.
• Added guarded shutdown that drains in-flight actions, cancels only bot-owned quotes, reconciles final fills, and unwinds only the job's virtual inventory.
• Isolated inventory and PnL by bot-owned order IDs, allowing existing positions and multiple algorithms on the same symbol without one bot flattening or cancelling another bot's state.
• Added optional percentage stop loss against allocated margin. It monitors job-owned realized PnL, midpoint-marked unrealized PnL, and signed fees or rebates, then cancels quotes, unwinds the job inventory, and terminates.
• Stopped refreshing the full account snapshot on every private WebSocket event. Account state is refreshed on demand and after reconnect, keeping order and fill handling off the HTTP path.
• Cached unchanged per-account leverage settings so rapid quote replacement does not submit a leverage update before every child order.
• Added bot-owned average-cost performance accounting with average buy and sell prices, matched volume, signed inventory and entry price, gross realized PnL, midpoint-marked unrealized PnL, signed fill fees or rebates, fee-adjusted trading PnL, and return on allocated margin.
• Persisted live performance on bot job heartbeats and exposed it through bot status, compact fill logs, structured JSON/JSONL events, and the terminal completion summary. Live fill logs mark open inventory at the current midpoint instead of the fill price. Funding is explicitly excluded, and fee-adjusted results become unavailable rather than inaccurate when a recovered fill has no fee data.
• Added the grid market maker with a fixed multi-level ladder anchored to the live touch, equal flat allocation, and automatic inventory-skewed replenishment.
• Added average-entry profit locking for inventory-reducing grid levels, flat-only full-range recentering, nonfatal maker-crossing recovery, and hard inventory headroom across resting and in-flight orders.
• Added an optional 0-1% soft-reset threshold that pauses the normal ladder and works only the residual inventory near midpoint after an adverse move, then reanchors after returning flat. The existing margin-based stop loss remains the terminal job-wide risk control.
• Added an ignored release benchmark for the local orderbook update and top-of-book hot paths.

CLI, Configuration, and Distribution

• Expanded marketlab.toml to cover sources, scripts, strategies, execution, and direct script parameters.
• Added consistent CLI-over-config precedence and validation.
• Added qualified script selectors to both repeated CLI flags and TOML arrays.
• Added source funding and source stats command surfaces.
• Added normalized terminal, JSON, and JSONL reporting across runtime and execution commands.
• Added market-lab as a library target alongside the mlab and mlabd binaries.
• Updated release packaging to ship both CLI and daemon binaries.
• Updated install and upgrade workflows for the expanded runtime.