BigQuery

Where the Sidecar fits in front of BigQuery: the Storage API on a grpc lane, and the REST plane’s boundaries

BigQuery exposes two planes, and they take different routes through the Sidecar. The Storage API (bigquerystorage.googleapis.com) is gRPC and fits a gRPC lane directly. The jobs and query API (bigquery.googleapis.com), the plane the bq CLI and the client libraries use for SQL, is REST over HTTPS; a gRPC lane never sees it, and the section below covers what an HTTP lane can and cannot do there.

The configurations on this page ride the grpc2 branch alongside the spanner protocol and have not shipped in a release yet.

---

The Storage API on a grpc lane

BigQueryRead streams table data as Arrow or Avro batches; BigQueryWrite appends rows. Method identity is the policy surface: the service and method travel in Tables, so a table rule fences the write plane with no new rule type.

config.yaml

listeners:
  - name: bqstorage
    protocol: grpc
    listen: 0.0.0.0:29060
    upstream: bigquerystorage.googleapis.com:443
    upstream_tls: {}
    guardrails:
      rules:
        - name: no-write-plane
          type: table
          tables: [google.cloud.bigquery.storage.v1.bigquerywrite]
          message: the write plane is fenced; this lane is read-only

The rule matches the lowercased service name. Clients get PERMISSION_DENIED with that message on AppendRows and reach CreateReadSession untouched.

With descriptors and capture_payload (the spanner page shows the buf command; add --path google/cloud/bigquery/storage), payload rules also see request fields. CreateReadSession names the table it opens, so the audit trail records who read which table. Row payloads stay opaque: ReadRows carries serialized Arrow batches inside a protobuf bytes field, and no descriptor turns those bytes into named columns. Content scanning and masking stop at that boundary.

---

The REST plane

BigQuery SQL travels as JSON over HTTPS to bigquery.googleapis.com. Two facts bound what a Sidecar lane can do with it today:

1. The http protocol is a relay: it does not terminate TLS toward the client and copies bytes as they are, Host header included. Google’s front end routes on Host, so a client pointed at the lane by address sends a Host the front end refuses. Working around that means overriding DNS for bigquery.googleapis.com on the client, which is a test-bench move.
2. The client’s OAuth token crosses the client-to-lane hop in cleartext.

Both are acceptable on loopback for a test bench and unacceptable anywhere else. This configuration exists for local validation:

config.yaml

listeners:
  - name: bigquery-rest        # loopback test bench only
    protocol: http
    listen: 127.0.0.1:29050
    upstream: bigquery.googleapis.com:443
    upstream_tls: {}
    http:
      capture_body: true       # jobs.insert carries the SQL in the JSON body

SQL-level policy for the REST plane, with the GoogleSQL lexer reading the query out of jobs.insert the way the spanner lane reads ExecuteSql, would need a dedicated extraction seam. The lexer dialect already exists; the seam does not.

---

The emulator

goccy/bigquery-emulator serves the REST plane on one port and the Storage gRPC plane beside it, BigQueryRead and BigQueryWrite included. We validated both directions through a licensed lane: AppendRows came back PERMISSION_DENIED with the fence rule’s message, and CreateReadSession crossed the lane and drew a real answer from the emulator about the named table.

The gcloud-stack compose file carries the emulator, a method-only bqstorage lane in front of it, and the write-plane rule shown above (commented out where the free tier’s one-rule budget is already spent).

---

Next

Spanner

The lane that reads GoogleSQL out of the RPC payloads.

HTTP

What the http relay captures, and the identity contract behind a proxy.