Runtime Safety
Understand Python process isolation, resource limits, failure behavior, and the trust boundary.
Each Python V2 session runs in a child process rather than inside the Rust process. If a hook loops forever, exhausts its allowed memory, or spawns too many children, Market Lab terminates the Python process group and fails the run without taking down mlabd.
Default limits
| Resource | Limit |
|---|---|
| Manifest inspection and process startup | 30 seconds |
Each on_data call | 5 seconds |
Each on_execution call | 5 seconds |
on_finish | 60 seconds |
| Aggregate Python process-tree memory | 2 GiB |
| Python process tree | 32 processes |
| One internal protocol message | 8 MiB |
| Cumulative Python stdout and stderr | 64 MiB |
The memory and process count cover the entire Python process group, including child processes created by packages. Monitoring continues between hooks, not only while a hook is returning.
When a hook-time, memory, process-count, or protocol limit is exceeded, Market Lab:
- stops accepting output from the hook;
- kills the Python process group;
- clears execution commands that were not completed by the failed hook; and
- records the failure in the script job and runtime report.
The log limit behaves differently: after 64 MiB, further Python output is discarded rather than terminating the run. These limits are fixed runtime safeguards in v0.0.8. They are not script parameters.
Infinite loops
This hook is terminated after five seconds:
def on_data(ctx, history):
while True:
passThe job fails with a timeout error. Later events are not delivered to a replacement Python process inside that same run.
Memory exhaustion
Memory is measured across the process tree. A script cannot bypass the limit by moving an allocation into a child process:
def on_data(ctx, history):
data = bytearray(3 * 1024 * 1024 * 1024)On Linux, Market Lab reads resident memory for processes in the Python process group through /proc. On macOS, it uses the operating system's physical-footprint accounting. Process-group termination and hook timeouts are supported on both platforms.
Logs and print
print(...), tracebacks, and normal Python stderr are forwarded into the Market Lab job logs:
mlab script logs <JOB_ID> --followThe Python bridge uses a separate internal channel, so normal printing does not corrupt runtime messages. After 64 MiB of cumulative Python log output, further output is discarded and Market Lab writes one truncation notice.
Failure behavior
An uncaught Python exception fails the current hook and includes its traceback in the Market Lab error. Because hooks are serialized, the runtime does not continue to the next event after a fatal hook failure.
on_finish runs when a backtest finishes or a live session ends through duration expiry, a stop request, SIGTERM, or an interrupt. It is not guaranteed after a process crash, a resource-limit kill, or an abrupt machine shutdown. Do not rely on it as the only place to persist critical state.
Live order signing, credentials, nonces, account streams, and execution journals remain inside mlabd. Delegated private keys are not serialized into the Python context or internal hook messages.
Trust boundary
Process isolation is a reliability boundary, not a security sandbox.
A Python script can use the current daemon user's permissions to:
- read and write accessible files;
- open network connections;
- inspect inherited environment variables;
- start subprocesses within the process-count limit; and
- import native extensions installed in its environment.
Run only code and dependencies you trust. With the Native backend, these permissions are those of the operating-system user running mlabd. With Docker, they are constrained by the daemon container, which still has intentional access to Market Lab state, credentials, and outbound networking. See Market Lab Daemon for the complete boundary.
Do not put secrets directly in a script, its parameters, logs, returned metadata, or artifacts.
Inspect runtime limits
Backtest and job reports identify the python engine and record hook counts, failures, durations, and the active Python limits. Use script status and structured output when diagnosing a slow or failed strategy:
mlab script status <JOB_ID> --output json
mlab script logs <JOB_ID> --output jsonl