> ## Documentation Index
> Fetch the complete documentation index at: https://docs.quanux.org/llms.txt
> Use this file to discover all available pages before exploring further.

# GCP data lake and DuckDB-to-BigQuery transpiler

> May 29, 2026 release: GCP historical data lake ingestion, DuckDB-to-BigQuery research transpiler, new quanuxctl infra commands, and fixes.

*May 29, 2026*

## New features

### GCP historical data lake (Tract 1)

QuanuX can now stream tick data from your live cluster into Google Cloud Storage as Parquet, with BigQuery External Tables registered automatically over the resulting datasets. This gives you a durable, queryable history of every market tick the engine has seen — no separate ETL job required.

* Bounded-memory ingestion: payloads flush to GCS before any memory ceiling is breached, so the pipeline is safe to run alongside latency-sensitive engines.
* BigQuery registration is a single command — point it at a bucket and you get a queryable external table.

See [Configuration → Secrets](/configuration/secrets) for the new `GCP_PROJECT_ID` and `GOOGLE_APPLICATION_CREDENTIALS` entries.

### DuckDB-to-BigQuery research transpiler (Tract 2)

Write research queries in DuckDB locally, then execute the same logic against your BigQuery historical lake with no rewrite. The transpiler parses your DuckDB query, validates it against an approved SQL surface, and emits BigQuery Standard SQL.

Supported in this release:

* `SELECT`, `FROM`, `WHERE`, `GROUP BY`, `ORDER BY`, `LIMIT`
* Aggregates: `COUNT`, `SUM`, `AVG`, `MIN`, `MAX`
* Bounded subqueries (scalar `SELECT`, `WHERE IN`, uncorrelated `FROM`) — nesting capped at depth 1
* Single inner-equality joins

Anything outside the approved matrix (outer joins, window functions, CTEs, mutations, correlated subqueries) is rejected with an explicit `TranspilationError` and direct-BigQuery fallback instructions — no silent semantic drift.

### `quanuxctl infra gcp-sql` command surface

New CLI subcommands for the research transpiler workflow:

* `quanuxctl infra gcp-sql validate` — check a query against the approved matrix without executing.
* `quanuxctl infra gcp-sql transpile` — emit BigQuery Standard SQL for a DuckDB query.
* `quanuxctl infra gcp-sql execute` — run the transpiled query against BigQuery with `--max-results`, `--timeout`, and `--dry-run` bounds.

Both human-readable and machine-readable JSON output formats are frozen for scripting. Exit codes follow a structured taxonomy (matrix rejection, auth, bounds, execution) so you can branch on failure reasons in CI.

See the [`quanuxctl` CLI reference](/cli/overview).

### `quanuxctl infra ingest-start` and `table-register`

New commands to start the JetStream → GCS ingestion pipeline and register a BigQuery External Table over a GCS bucket of Parquet files.

## Updates

* **Secrets manager** now recognizes `GCP_PROJECT_ID` and `GOOGLE_APPLICATION_CREDENTIALS`, resolved automatically from the OS keyring when environment variables are absent. See [Secrets](/configuration/secrets).
* **`quanuxctl infra`** now accepts a provider flag so the same command group serves multi-cloud deployments.
* **Dependency refresh**: `hono`, `react-hook-form`, `lucide-react`, `three`, `@modelcontextprotocol/sdk`, `@tauri-apps/cli`, `graphql`, and `figma-developer-mcp` have all been bumped to their latest patch and minor releases.

## Bug fixes

* Fixed a memory-overshoot edge case in the ingestion pipeline where a single high-velocity influx event could briefly exceed the configured bound before flushing.
* Tightened credential resolution in `quanuxctl infra gcp-sql` so missing GCP environment variables stop the pipeline immediately with a clear error, instead of failing partway through execution.