Skip to content

Commit 207d641

Browse files
committed
feat: add Kysely queue adapter
1 parent 757055a commit 207d641

11 files changed

Lines changed: 1423 additions & 51 deletions

.yarnrc.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
approvedGitRepositories:
2-
- "**"
2+
- '**'
33

44
enableScripts: true
55

README.md

Lines changed: 41 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ npm install @boringnode/queue
2020

2121
## Features
2222

23-
- **Multiple Queue Adapters**: Redis, Knex (PostgreSQL, MySQL, SQLite), and Sync
23+
- **Multiple Queue Adapters**: Redis, Knex, Kysely (PostgreSQL, MySQL, SQLite), and Sync
2424
- **Type-Safe Jobs**: TypeScript classes with typed payloads
2525
- **Delayed Jobs**: Schedule jobs to run after a delay
2626
- **Priority Queues**: Process high-priority jobs first
@@ -267,18 +267,18 @@ const { jobId, deduped } = await SaveDraftJob.dispatch({ content: '...' })
267267
- `retryJob` does not touch the dedup entry — a retried job continues to occupy the dedup slot. TTL runs on wall-clock time, so long-running retries may outlive the TTL window. Use a generous TTL or no TTL if retries must stay deduped.
268268
- Atomic and race-free:
269269
- **Redis**: a single Lua script per dispatch performs the dedup-key lookup, state check (pending/delayed ZSCORE), payload swap, and TTL refresh atomically.
270-
- **Knex**: transactional `SELECT ... FOR UPDATE` + insert/update inside a transaction. A nested savepoint catches unique-constraint violations under concurrent inserts and returns `{ deduped: 'skipped' }` pointing at the winner.
270+
- **Knex/Kysely**: transactional `SELECT ... FOR UPDATE` + insert/update inside a transaction. A savepoint catches unique-constraint violations under concurrent inserts and returns `{ deduped: 'skipped' }` pointing at the winner.
271271
- **SyncAdapter**: executes inline, no dedup support.
272272

273273
### Caveats
274274

275275
- Without `.dedup()`, jobs use auto-generated UUIDs and are never deduplicated.
276-
- The **Sync adapter** ignores `.dedup()` entirely — every dispatch executes inline and `deduped` is always `undefined` on the result. Use Redis or Knex if you need real deduplication.
276+
- The **Sync adapter** ignores `.dedup()` entirely — every dispatch executes inline and `deduped` is always `undefined` on the result. Use Redis, Knex, or Kysely if you need real deduplication.
277277
- `.dedup()` is only available on single dispatch. `dispatchMany` / `pushManyOn` reject jobs with a `dedup` field.
278278
- Scheduled jobs (`.schedule()`) do not support dedup — each cron/interval fire is an independent dispatch.
279279
- With no `ttl`, dedup persists until the job is removed (completed/failed without retention). When retention keeps the record, re-dispatch stays blocked until the record is pruned.
280280
- With `ttl`, dedup expires after the window — a new job (new UUID) is created. The old job still runs.
281-
- Knex MySQL concurrent race: MySQL does not support partial unique indexes, so two `pushOn` calls with the same dedup id firing at the exact same instant can both succeed. Serialize at the app layer if strict guarantees are required, or use Postgres / SQLite / Redis (all of which serialize correctly via the partial unique index or Lua atomicity).
281+
- Knex/Kysely MySQL concurrent race: MySQL does not support partial unique indexes, so two `pushOn` calls with the same dedup id firing at the exact same instant can both succeed. Serialize at the app layer if strict guarantees are required, or use Postgres / SQLite / Redis (all of which serialize correctly via the partial unique index or Lua atomicity).
282282

283283
## Job History & Retention
284284

@@ -405,6 +405,43 @@ export default class extends BaseSchema {
405405

406406
</details>
407407

408+
### Kysely (PostgreSQL, MySQL, SQLite)
409+
410+
Pass the application-owned Kysely instance to the adapter factory.
411+
412+
```typescript
413+
import {
414+
kysely,
415+
KyselyQueueSchemaService,
416+
type QueueDatabase,
417+
} from '@boringnode/queue/drivers/kysely_adapter'
418+
419+
interface Database extends QueueDatabase {
420+
orders: OrderTable
421+
}
422+
423+
const adapter = kysely<Database>(db, { dialect: 'postgres' })
424+
```
425+
426+
The adapter never destroys the application-owned Kysely instance. Use `tableName` and
427+
`schedulesTableName` in the options when your migration uses custom names.
428+
429+
<details>
430+
<summary><strong>Database setup with KyselyQueueSchemaService</strong></summary>
431+
432+
```typescript
433+
const schema = new KyselyQueueSchemaService(db, { dialect: 'postgres' })
434+
435+
await schema.createJobsTable()
436+
await schema.createSchedulesTable()
437+
438+
// down
439+
await schema.dropSchedulesTable()
440+
await schema.dropJobsTable()
441+
```
442+
443+
</details>
444+
408445
### Fake (testing + assertions)
409446

410447
```typescript

compose.yml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,11 @@ services:
1212
POSTGRES_USER: postgres
1313
POSTGRES_PASSWORD: postgres
1414
POSTGRES_DB: queue_test
15+
16+
mysql:
17+
image: mysql:8.4
18+
ports:
19+
- '3307:3306'
20+
environment:
21+
MYSQL_ROOT_PASSWORD: mysql
22+
MYSQL_DATABASE: queue_test

package.json

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,8 @@
5757
"hot-hook": "^1.0.0",
5858
"ioredis": "^5.11.1",
5959
"knex": "3.1.0",
60+
"kysely": "^0.29.2",
61+
"mysql2": "^3.0.0",
6062
"oxfmt": "^0.56.0",
6163
"oxlint": "^1.71.0",
6264
"oxlint-tsgolint": "^0.17.4",
@@ -70,7 +72,8 @@
7072
"@opentelemetry/core": "^1.30.0 || ^2.0.0",
7173
"@opentelemetry/instrumentation": "^0.200.0",
7274
"ioredis": "^5.0.0",
73-
"knex": "^3.0.0"
75+
"knex": "^3.0.0",
76+
"kysely": "^0.29.0"
7477
},
7578
"peerDependenciesMeta": {
7679
"@opentelemetry/api": {
@@ -87,6 +90,9 @@
8790
},
8891
"knex": {
8992
"optional": true
93+
},
94+
"kysely": {
95+
"optional": true
9096
}
9197
},
9298
"author": "Romain Lanz <romain.lanz@pm.me>",

src/contracts/adapter.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ export interface AcquiredJob extends JobData {
3232
* Adapter interface for queue storage backends.
3333
*
3434
* Implementations handle job persistence, atomic operations, and
35-
* concurrency control. Built-in adapters: Redis, Knex (PostgreSQL/SQLite).
35+
* concurrency control. Built-in adapters: Redis, Knex, Kysely, and Sync.
3636
*
3737
* @example
3838
* ```typescript

0 commit comments

Comments
 (0)