UltraTab

Ultra-fast native CSV and XLSX streaming parser for Node.js. Built as a C++ addon with SIMD acceleration, background-thread parsing, and bounded memory for large files (100MB–10GB+).

Features

• Streaming: Parses from disk in chunks; does not load entire files into memory
• Non-blocking: Parsing runs on a C++ background thread; the Node event loop stays responsive
• Two APIs: Row-based csv() (string[][]) and typed columnar csvColumns() (TypedArrays)
• Typed output: int32, int64, float64, bool → Int32Array, BigInt64Array, Float64Array, Uint8Array
• XLSX support: xlsx() parses .xlsx files in low-memory streaming mode
• SIMD acceleration: AVX2/SSE2 on x86_64 (Linux/Windows); scalar fallback on macOS

Installation

npm install ultratab

Requirements: Node.js >= 18, CMake >= 3.15, C++17 compiler (GCC, Clang, or MSVC)

UltraTab is written in TypeScript and ships with full type definitions. Use import (ESM) or require() (CommonJS).

Quick Start

After npm install ultratab, try the example:

node node_modules/ultratab/examples/csv-demo.js

CSV (row-based)

import { csv } from "ultratab";

for await (const batch of csv("data.csv", { batchSize: 10000, headers: true })) {
  console.log("Rows in batch:", batch.length);
  for (const row of batch) {
    console.log(row); // string[]
  }
}

CSV (typed columnar)

import { csvColumns } from "ultratab";

for await (const batch of csvColumns("data.csv", {
  headers: true,
  schema: { id: "int32", amount: "float64", active: "bool" },
  select: ["id", "amount", "active"],
})) {
  const ids = batch.columns.id;       // Int32Array
  const amounts = batch.columns.amount; // Float64Array
  const nulls = batch.nullMask?.amount; // 1 = null
  for (let i = 0; i < batch.rows; i++) {
    if (nulls?.[i]) continue;
    console.log(ids[i], amounts[i]);
  }
}

XLSX

import { xlsx } from "ultratab";

for await (const batch of xlsx("data.xlsx", {
  sheet: 1,
  headers: true,
  batchSize: 5000,
})) {
  console.log("Headers:", batch.headers);
  console.log("Rows:", batch.rowsCount);
}

API Reference

csv(path, options?)

Returns AsyncIterable<string[][]>. Each batch is an array of rows; each row is string[].

Option	Type	Default	Description
delimiter	string	","	Field delimiter (use "\t" for TSV)
quote	string	'"'	Quote character
headers	boolean	false	Skip first row as header
batchSize	number	10000	Rows per batch (1–10,000,000)
maxQueueBatches	number	2	Max batches in queue (backpressure)
useMmap	boolean	false	Use memory-mapped I/O
readBufferSize	number	262144	Read buffer size in bytes

csvColumns(path, options?)

Returns AsyncIterable<ColumnarBatch>. Each batch has headers, columns, nullMask, and rows.

Option	Type	Default	Description
delimiter	string	","	Field delimiter
quote	string	'"'	Quote character
headers	boolean	true	First row is header
batchSize	number	10000	Rows per batch
select	string[]	(all)	Columns to keep by header name
schema	object	(string)	Per-column: "string", "int32", "int64", "float64", "bool"
nullValues	string[]	["","null","NULL"]	Strings treated as null
trim	boolean	false	Trim whitespace
typedFallback	string	"null"	On parse failure: "null" or "string"

xlsx(path, options?)

Returns AsyncIterable<XlsxBatchResult>. Options: sheet, headers, batchSize, select, schema, nullValues, trim, typedFallback.

Performance

Designed for large files: minimal allocations, SIMD-accelerated scanning (x86_64), and bounded backpressure. Typical throughput: hundreds of thousands to millions of rows per second depending on schema and hardware.

Platform Compatibility

• Linux (x86_64): Full SIMD (AVX2/SSE2)
• Windows (x64): Full SIMD
• macOS (Intel/ARM): Scalar path (no SIMD flags; still fast)

Build from Source

git clone https://github.com/haykghukasyan/ultratab.git
cd ultratab
npm install
npm run build

Compiler requirements:

• Windows: Visual Studio Build Tools
• macOS: Xcode Command Line Tools (xcode-select --install)
• Linux: build-essential (or equivalent)

Troubleshooting

"Native addon not found"

• Run npm run build or npm rebuild ultratab
• Ensure CMake and a C++17 compiler are installed

Build fails on Windows

• Install Visual Studio Build Tools with the "Desktop development with C++" workload

Build fails on Linux

• sudo apt-get install cmake build-essential (Debian/Ubuntu)

License

MIT