huge-async-storage

A robust wrapper around React Native's AsyncStorage that enables storage of large data exceeding typical size limitations by intelligently chunking the data.

Features

• Store unlimited data size: Automatically splits data into 1MB chunks
• TypeScript support: Full type safety with generics
• Promise-based API: Modern async/await syntax
• Error handling: Comprehensive error messages for debugging
• Idempotent operations: Safe to call multiple times
• Cleanup on failure: Automatically removes partial data on storage errors
• Null-safety: Handles missing or corrupted data gracefully

Installation

yarn add huge-async-storage
# or
npm install huge-async-storage

Quick Start

import { storeAsync, getAsync, removeAsync } from "huge-async-storage";

// Store large data (automatically chunked)
await storeAsync('users', { list: Array(1000000).fill({ id: 1, name: 'John' }) });

// Retrieve the data
const users = await getAsync('users');

// Remove the data
await removeAsync('users');

API Reference

storeAsync<T>(key: string, data: T): Promise<void>

Stores data by splitting it into manageable chunks and saving them to AsyncStorage.

Parameters

Parameter	Type	Required	Description
key	string	Yes	The storage key. Must be non-empty and not whitespace-only.
data	T	Yes	The data to store. Will be JSON serialized.

Throws

• Error - If key is empty or whitespace-only
• Error - If data cannot be serialized (e.g., circular references)
• Error - If storage operation fails (with automatic cleanup of partial data)

Side Effects

• Creates multiple keys in AsyncStorage: key0, key1, ..., keyN for data chunks
• Stores chunk count under the original key
• On failure, removes all partially written chunks

Examples

// Simple object
await storeAsync('user', { name: 'Alice', age: 30 });

// Large array (will be chunked automatically)
const largeData = { items: Array(1000000).fill('data') };
await storeAsync('large', largeData);

// Null values are supported
await storeAsync('settings', null);

// Complex nested objects
await storeAsync('config', {
  database: { host: 'localhost', port: 5432 },
  features: ['auth', 'logging', 'caching']
});

---

getAsync<T>(key: string): Promise<T>

Retrieves and reconstructs data that was stored using storeAsync.

Parameters

Parameter	Type	Required	Description
key	string	Yes	The storage key to retrieve.

Returns

Promise<T> - The reconstructed data with the original type.

Throws

• Error - If key is empty or whitespace-only
• Error - If no data exists for the given key
• Error - If chunk count is invalid or corrupted
• Error - If any chunk is missing (storage corruption)
• Error - If data cannot be parsed as valid JSON

Examples

// With explicit type
interface User {
  name: string;
  age: number;
  active: boolean;
}

const user = await getAsync<User>('user');
console.log(user.name); // Type-safe access

// Type inference
const settings = await getAsync<{ theme: 'light' | 'dark' }>('settings');

// Array types
const items = await getAsync<number[]>('numbers');
items.map(n => n * 2); // Type-safe operations

---

removeAsync(key: string): Promise<void>

Removes all chunks and metadata associated with a key.

Parameters

Parameter	Type	Required	Description
key	string	Yes	The storage key to remove.

Throws

• Error - If key is empty or whitespace-only
• Error - If chunk count is invalid
• Error - If removal operation fails

Side Effects

• Removes all chunk keys (key0, key1, ..., keyN)
• Removes the metadata key

Behavior

• Idempotent: Calling removeAsync multiple times on the same key is safe
• Silent success: If the key doesn't exist, the operation succeeds without error

Examples

// Remove stored data
await removeAsync('tempData');

// Safe to call multiple times
await removeAsync('cache');
await removeAsync('cache'); // No error thrown

// Non-existent keys are handled gracefully
await removeAsync('neverStored'); // Success, no error

---

How It Works

Chunking Strategy

1. Data is serialized to JSON using JSON.stringify()
2. The serialized string is split into chunks of 1,000,000 characters
3. Each chunk is stored with a numeric suffix: key0, key1, key2, etc.
4. The total number of chunks is stored under the original key

Example Flow

storeAsync('myData', largeObject)
  ↓
JSON.stringify(largeObject) → '{"items":[...]}' (2.5M chars)
  ↓
Split into chunks:
  - myData0: 1,000,000 chars
  - myData1: 1,000,000 chars
  - myData2: 500,000 chars
  ↓
Store chunk count:
  - myData: "3"

Storage Layout

AsyncStorage Keys:
┌─────────────────────────────────────┐
│ myData    → "3" (chunk count)       │
│ myData0   → chunk 1 (1MB)           │
│ myData1   → chunk 2 (1MB)           │
│ myData2   → chunk 3 (remaining)     │
└─────────────────────────────────────┘

Error Handling

All functions provide detailed error messages to help diagnose issues:

try {
  await getAsync('corrupted');
} catch (error) {
  // Error: Storage corruption: chunk 2 of 5 missing for key "corrupted"
  console.error(error.message);
}

Common Errors

Error	Cause	Solution
Storage key cannot be empty	Empty or whitespace key	Provide a valid key
No data found for key "x"	Key doesn't exist	Check if data was stored
Invalid chunk count for key "x"	Corrupted metadata	Remove and re-store data
Storage corruption: chunk N of M missing	Partial data loss	Remove and re-store data
Failed to parse data for key "x"	Invalid JSON	Check data integrity

Best Practices

1. Use Type Guards

interface UserProfile {
  id: string;
  name: string;
  email?: string;
}

const user = await getAsync<UserProfile>('user');
if (user?.email) {
  sendEmail(user.email);
}

2. Handle Errors Gracefully

async function loadConfig() {
  try {
    const config = await getAsync<Config>('config');
    return config ?? defaultConfig;
  } catch (error) {
    console.warn('Failed to load config, using defaults');
    return defaultConfig;
  }
}

3. Clean Up Unused Data

// Remove old data when no longer needed
await removeAsync('tempCache');

4. Avoid Very Large Keys

Short keys are more efficient:

// Good
await storeAsync('usr', userData);

// Avoid
await storeAsync('veryLongKeyNameThatWastesMemory', userData);

Limitations

• Chunk Size: Fixed at 1MB per chunk for compatibility across platforms
• Synchronous Operations: Each operation is atomic; concurrent writes to the same key may conflict
• Memory Usage: Retrieving very large data loads everything into memory

Platform Support

Platform	Total Storage	Per-Entry Limit	Chunk Size	Notes
iOS	~6MB default	~6MB	1MB	Safe within default limits
Android	~6MB default (configurable)	~2MB (WindowCursor SQLite)	1MB	✅ Below Android's 2MB per-entry limit

Why 1MB Chunk Size?

The 1MB chunk size is specifically designed to work within Android's WindowCursor SQLite limit of approximately 2MB per entry:

"Per-entry is limited by a size of a WindowCursor, a buffer used to read data from SQLite. Currently it's size is around 2 MB." — AsyncStorage Documentation

By using 1MB chunks, this library:

• ✅ Stays safely below Android's 2MB per-entry limit
• ✅ Allows storing data larger than the 6MB total limit through chunking
• ✅ Works across iOS and Android without platform-specific code

License

MIT