feat: Add Arrow Native (ADBC) Server Protocol

[borodark]

Check List

• Tests have been run in packages where changes have been made if available
• Linter has been run for changed code
• Tests for the changes have been added if not covered yet
• Docs have been added / updated if required

Adds an Arrow Native server to CubeSQL that speaks Arrow IPC protocol on port 8120, enabling 8-15x faster data transfer compared to the REST HTTP API.

Closes #10296

What this PR does

• Arrow Native server on configurable port (default: 8120)
• Binary Arrow IPC protocol - no JSON serialization overhead
• Optional query result cache - additional 3-10x speedup on repeated queries
• Works with any ADBC client - Python, Elixir, R, etc.

Architecture

  Client (Python/Elixir/R via ADBC)
           │
           ├─── REST HTTP (Port 4008) - existing
           │    └─> JSON serialization → Cube API
           │
           └─── Arrow Native (Port 8120) - NEW
                └─> Binary Arrow IPC
                     └─> Optional Results Cache
                          └─> Cube API

  Performance

  | Query Size | Arrow Native | REST API | Speedup |
  |------------|--------------|----------|---------|
  | 200 rows   | 42ms         | 1414ms   | 33x     |
  | 2K rows    | 2ms          | 1576ms   | 788x    |
  | 20K rows   | 8ms          | 2133ms   | 266x    |

Configuration

  # Enable Arrow Native server (enabled by default when port is set)
  CUBEJS_ADBC_PORT=8120

  # Optional query result cache
  CUBESQL_ARROW_RESULTS_CACHE_ENABLED=true      # default: true
  CUBESQL_ARROW_RESULTS_CACHE_MAX_ENTRIES=1000  # default: 1000
  CUBESQL_ARROW_RESULTS_CACHE_TTL=3600          # default: 3600s

Files Changed

Core Implementation (rust/cubesql/cubesql/src/):

• sql/arrow_native/server.rs - Arrow Native server
• sql/arrow_native/protocol.rs - Wire protocol
• sql/arrow_native/stream_writer.rs - Arrow IPC streaming
• sql/arrow_native/cache.rs - Query result cache
• config/mod.rs - Configuration and DI

Integration:

• packages/cubejs-backend-shared/src/env.ts - Environment variables
• packages/cubejs-server-core/ - Server initialization
• docs/ - Environment variable documentation

Example (examples/recipes/arrow-ipc/):

• Complete working example with Python tests
• Sample data (3000 orders)
• Performance benchmarks

Testing

  # Unit tests
  cd rust/cubesql
  cargo test arrow_native

  # Integration test with example
  cd examples/recipes/arrow-ipc
  docker-compose up -d postgres
  ./setup_test_data.sh
  ./start-cube-api.sh &
  ./start-cubesqld.sh &
  python test_arrow_native_performance.py

Ecosystem Compatibility

Tested with:

• Python: python client
• Elixir: https://github.com/livebook-dev/adbc with feat: Add Cube ADBC Driver for CubeSQL  borodark/adbc#2
• Real-world usage: DataFrame from ADBC Client of Cube  borodark/power_of_three#5 Elixir library

Checklist

• Code compiles without warnings (cargo clippy)
• Code is formatted (cargo fmt)
• Unit tests pass (cargo test)
• Example works end-to-end
• Documentation updated
• No breaking changes to existing APIs