> ## Documentation Index
> Fetch the complete documentation index at: https://docs.quanux.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Backtest strategies with the Crucible engine

> Run strategies through Crucible, QuanuX's C++20 DuckDB-backed engine — far faster than pandas backtesters, with institutional-grade L3 execution metrics.

Crucible is QuanuX's backtesting engine, written in C++20. It executes your Python and Cython strategies at native speeds by bypassing standard memory copies, SQL overhead, and Python object serialization entirely. Under the hood, Crucible uses the native `duckdb::Appender` API to inject vectorized blocks of trade data directly into DuckDB's in-memory columnar engine, with the Python boundary bridged exclusively via Cython. The result is throughput approaching 100x that of traditional pandas-based backtesters.

You control Crucible entirely through `quanuxctl`. The engine runs as an isolated background process so it does not block your terminal or compete with other research workloads.

## Before you start

Crucible expects a Foundry-generated test harness to exist at `server/backtests/<strategy>_v<version>/` before you run `start`. If that directory is missing, generate your strategy with the Foundry first, or create the harness manually following the expected layout.

<Note>
  Crucible writes its state to `server/backtests/<strategy>_v<version>/crucible.duckdb`. Do not delete or modify this file while the engine is running — doing so will produce a fatal segmentation fault in the C++ appender.
</Note>

## Backtesting workflow

<Steps>
  <Step title="Start the engine">
    Launch a Crucible simulation for your strategy. Replace `my_strategy` with your strategy name and supply the version string that matches the Foundry-generated harness.

    ```bash theme={null}
    quanuxctl crucible start my_strategy --version 1.0.0
    ```

    Crucible forks from the terminal via `os.setsid()` and writes the OS process ID to `/tmp/quanux_crucible.pid`. The simulation runs entirely in the background.
  </Step>

  <Step title="Monitor resource usage">
    Check CPU and memory consumption while the simulation is running.

    ```bash theme={null}
    quanuxctl crucible status
    ```

    `status` reads the PID from `/tmp/quanux_crucible.pid` and uses `psutil` to return real-time CPU load and RAM consumption inside the C++ 128-byte aligned memory pools.

    For live streaming telemetry, subscribe to the NATS topic directly:

    ```bash theme={null}
    nats sub "sys.crucible.report.my_strategy"
    ```
  </Step>

  <Step title="Pull the metrics report">
    Extract the instantaneous metrics payload from the running engine.

    ```bash theme={null}
    quanuxctl crucible report my_strategy
    ```

    `report` bypasses the Python pandas pipeline entirely. It loads the compiled Cython extension from `QuanuX-Backtesting-Engine/python` and calls the native `DuckDBFeeder::get_metrics_json()` method directly through the C++ DuckDB API, reading from the live `.duckdb` state file.

    You can also target a specific version:

    ```bash theme={null}
    quanuxctl crucible report my_strategy --version 1.0.0
    ```
  </Step>

  <Step title="Stop the engine">
    Gracefully terminate the simulation when you are done.

    ```bash theme={null}
    quanuxctl crucible stop
    ```

    `stop` sends a POSIX `SIGTERM` to the engine PID and waits five seconds. If the process has not exited after that window, it sends `SIGKILL` to halt runaway algorithms.
  </Step>
</Steps>

## L3 execution metrics

Unlike basic backtesters that only track P\&L, Crucible enforces institutional-grade L3 (Market By Order) metric collection natively. Every trade record is stored as a `CrucibleTrade` struct aligned to 128 bytes for optimal CPU cache line saturation.

| Metric                      | Description                                                                                                        |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| **Latency Slippage (bps)**  | Slippage measured in basis points, calculated automatically from execution delay relative to the signal timestamp. |
| **Queue Position at Entry** | Estimated position in the order queue, determined by simulated volume-ahead tracking in the `FifoMatcher`.         |
| **MAE**                     | Maximum Adverse Excursion — the worst intra-trade drawdown from entry price, tracked per tick.                     |
| **MFE**                     | Maximum Favorable Excursion — the best intra-trade unrealized gain from entry price, tracked per tick.             |

<Tip>
  MAE and MFE together give you a clear picture of whether your exit logic is leaving money on the table (high MFE, low capture) or cutting losses late (high MAE). Review these alongside win rate before promoting a strategy to C++.
</Tip>

## Backtest state files

Crucible stores all persistent state in the following locations:

| Path                                                     | Contents                                                                                                             |
| -------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| `server/backtests/<strategy>_v<version>/crucible.duckdb` | Live DuckDB state file written natively by the C++ engine. Contains all `CrucibleTrade` records for the current run. |
| `/tmp/quanux_crucible.pid`                               | OS process ID of the currently running simulation. Cleared automatically on `stop`.                                  |

You can query the `.duckdb` file directly with the DuckDB CLI after stopping the engine for ad-hoc analysis:

```bash theme={null}
duckdb server/backtests/my_strategy_v1.0.0/crucible.duckdb \
  "SELECT symbol, COUNT(*) as trades, AVG(latency_slippage_bps) as avg_slip FROM crucible_trades GROUP BY symbol"
```

## Architecture note

Crucible's `CrucibleTrade` struct is explicitly aligned to 128 bytes to maintain memory locality inside CPU L1 and L2 data caches. If you are extending Crucible and modify this struct, you must propagate the changes down to the Cython header at `QuanuX-Backtesting-Engine/python/quanux_crucible.pxd`. Mismatches between the C++ struct and the Cython header produce fatal segmentation faults at runtime.