Releases: go-again/sqlite
v0.6.0
Highlights
- SESSION extension typed API — record changes on a primary, serialize as a changeset blob, apply onto a replica with typed conflict handlers and table filters; invert and concat included.
- WAL / snapshot / progress / db-config control — typed
(*Conn).WALCheckpoint,RegisterWALHook,GetSnapshot/OpenSnapshot,SetProgressHandler,SetDBConfig/QueryDBConfig. - R-Tree custom-geometry / custom-query callbacks —
(*Conn).RegisterRTreeGeometryandRegisterRTreeQuerylet Go code drive the R-Tree MATCH operator;ext/rtree.Tableis the typed handle on top. - Six new typed vtab handles —
spellfix1.Vocab,bloom.Filter,closure.Graph,csv.Table,lines.Table,rtree.Table. All follow thevec.Table/fts.Indexshape:Create/Open/ domain methods /Drop,WithIfNotExists+ErrAlreadyExists. sqlitex/utility package —Migrate(ctx, db, fsys)over anfs.FSofNNNN_name.sqlfiles;Save(savepoint with deferred-error closure);Transaction/ImmediateTransaction;ExecScript;ResultInt/ResultText/ResultFloat/ResultBoolsingle-value helpers.
New features
SESSION extension (session.go, snapshot.go)
(*Conn).CreateSession(schema)→*SessionwithAttach(empty = all tables),Enable/IsEnabled/IsEmpty,Changeset/Patchsetserializers,Diff(fromSchema, table)for two-DB diffing,Closefinalizer.(*Conn).InvertChangeset(cs),ConcatChangesets(a, b)— top-level transforms.(*Conn).ApplyChangeset(cs, opts...)withWithConflictHandler(h)(typedConflictType+ConflictActionenums) andWithTableFilter(f). Default (no handler) aborts and rolls back.(*Conn).GetSnapshot(schema)/OpenSnapshot(schema, snap)/SnapshotRecover(schema)for WAL point-in-time reads;(*Snapshot).Cmp/Close.- Coverage matrix:
docs/coverage-session.md.
WAL + progress + db-config (wal.go, control.go)
(*Conn).WALCheckpoint(schema, mode)returns(logFrames, checkpointed)with typedCheckpointMode(Passive / Full / Restart / Truncate).(*Conn).WALAutoCheckpoint(frames),RegisterWALHook(hook).(*Conn).SetProgressHandler(n, handler)for periodic-VM-instruction interrupt callbacks.(*Conn).SetDBConfig(op, enable)/QueryDBConfig(op)oversqlite3_db_configwith typedDBConfigOpenum — including the security-onlyDEFENSIVE/TRUSTED_SCHEMA/WRITABLE_SCHEMAflags that have no PRAGMA equivalent.
Introspection (introspect.go, stmt.go)
(*Conn).TableColumnMetadata(schema, table, column)returnsColumnMetadata{decltype, collation, notnull, pk, autoinc}.(*Conn).Status(op DBStatus, reset)— per-conn cache / lookaside counters.(*Conn).TxnState(schema)— typedTxnState(None / Read / Write).(*Stmt).Readonly()— read/write routing viasqlite3_stmt_readonly.(*Stmt).Status(op StmtStatus, reset)— per-statement VM-step / sort / fullscan counters.- Coverage matrix:
docs/coverage-conn.md.
R-Tree custom callbacks (rtree.go)
RTreeGeometryFunc(coords, params) (overlap, err)— pruning callback.RTreeQueryFunc(*RTreeQueryInfo)— richer per-node form with tree depth + parent classification.- Typed
RTreeWithinenum (NotWithin / PartlyWithin / FullyWithin);RTreeQueryInfocarriesCoords,Params,Level,MaxLevel,Rowid,ParentScore,ParentWithin.
Typed vtab handles
Six new entry points, each Create / Open / domain ops / Drop with WithIfNotExists + ErrAlreadyExists:
spellfix1.Vocab— Add / AddMany / Size / Correct / CorrectSQL withWithMaxDistance/WithLimit. UNIQUE index onwordso re-adding dedupes via INSERT OR IGNORE.bloom.Filter— Add / AddMany / Contains withWithSize/WithFalsePositiveRate/WithHashes.closure.Graph— Descendants / DescendantsSQL withWithMaxDepth/WithExactDepth/OverGraphper-query retarget;Reversed(over)for ancestor walks.csv.Table— Columns / Rows over the CSV vtab;WithFilename/WithData/WithHeader/WithComma/WithComment/WithColumns.lines.Table— Columns / Rows (lineno + line) over the one-row-per-line vtab.rtree.Table— Insert / InsertPoint / Delete / Search / SearchCircle withWithDimensions(n)/WithInt32Coords().
sqlitex/ utility helpers
LoadMigrations(fsys)— parseNNNN_name.sqlfiles (strict numeric prefix; bounded to user_version's int32 range).Migrate(ctx, db, fsys)— apply pending migrations idempotently against aschema_migrationstracking table.Save(ctx, conn)— savepoint with a deferred-error release closure sodefer release(&err)works.Transaction(ctx, db, fn)andImmediateTransaction(ctx, conn, fn)— auto commit/rollback wrappers (Immediate takes the write lock up front).ExecScript(ctx, e, script)— multi-statement SQL.ResultInt/ResultText/ResultFloat/ResultBool— collapse the QueryRow + Scan dance for single-value queries.
New ext sub-packages
ext/series—generate_series-style eponymous vtab.ext/text—text_reverse,text_repeat,text_lpad,text_rpad,text_splitscalars absent from core.ext/encode—encode(data, format)/decode(text, format)covering hex / base32 / base64 + URL-safe variants.
Improvements
(*conn).Closenow drains the rtree-geom, rtree-query, progress-handler, and WAL-hook registries alongside the prior six (update / authorizer / trace / pre-update / commit / rollback). The expandeddropHookHandlersdoc enumerates every per-conn registry and notes that future ones MUST be drained here.- The CREATE VIRTUAL TABLE skeleton in each typed vtab handle moves to
internal/vtabx.Create/Drop— name validation, IF NOT EXISTS, parameter assembly,ErrAlreadyExistsmapping in one place. - A generic
callbackTable[T]lifts the{mu, m, ids}shape every per-conn C-callback registry rolled by hand; changeset apply, rtree geometry, and rtree query all use it. internal/sqlidgrowsQuoteString(single-quoted SQL literal) consolidated fromext/csvandext/lines;QuoteIdent/QuoteIdentBacktick/QuoteStringnow spell out their NUL-byte precondition in the docstring.
Bug fixes
- NUL-byte safety on SQL emission —
fts.Newrejects NUL bytes in any tokenizer field (Tokenchars,Separators,Categories) before they truncate the generated SQL at the libc.CString boundary;wrapTokenizedoubles embedded single quotes to close the outertokenize='…'string-literal breakout.ext/csvfalls back to the positional column name when a CSV header cell contains a NUL. - pre-update accessor honesty —
(*SQLitePreUpdateData).valuezeros the output slot beforesqlite3_preupdate_new/_old(Xmalloc does not zero), honors the returned rc instead of dereferencing whatever sat in memory, and surfaces an absent OLD/NEW column as(nil, nil). - vtab xCreate NOMEM cleanup — when
sqlite3_mallocfor the vtab struct fails, the Go-side Table'sDestroy()runs explicitly so any shadow tables / handles allocated inCreateare released. Previously SQLite never got a handle to drivexDestroyagainst on that branch. (*Conn).OpenBloband(*Blob).Reopenoutput slots live in C memory, not on the Go stack. A Go stack address passed as untrackeduintptrcould go stale when the runtime moves the stack (reentrant calls deep in the graph), leavingsqlite3_blob_openwriting through a stale pointer. Same fix applied to(*conn).loadExtension.- UDF embedded-NUL TEXT path —
functionArgsand the pre-update TEXT case droplibc.GoStringforXsqlite3_value_text+Xsqlite3_value_bytes+ an explicit length-bounded copy, so"foo\x00bar"survives intact instead of truncating at the first NUL. Accessor order also swapped to the documented-safe form (text/blob content accessor beforevalue_bytes). - UDF result > 2 GiB rejected with
sqlite3_result_error_toobiginstead of silently wrappingint32(len)to a negative size. sqlitex.parseMigrationNamerejects leading+/-in the version prefix and bounds the parsed version tomath.MaxInt32(thePRAGMA user_versionrange) so a truncated round-trip doesn't makeMigratere-apply the migration on every run.
Internal / dev
internal/vtabxconsolidates the typed-vtab CREATE/DROP skeleton;callback_table.goconsolidates the per-conn C-callback registry shape;internal/sqlidgrowsQuoteString.docs/coverage-conn.mdanddocs/coverage-session.mdare new coverage matrices covering the connection-level and SESSION surfaces.- The drift-discipline note in
CLAUDE.mdandvfs/crypto/doc.gois sharpened: named-field struct literals catch renames / removals at compile time, but not additions or reordering — safety against added fields comes from hard-codingFiVersion/iVersionbelow any unforwarded field.
Dependencies
modernc.org/sqlitebumped (carries the SQLite engine pin).golang.org/x/crypto,golang.org/x/text,golang.org/x/syscarried bygo mod tidy.modernc.org/libcstays on the version the new sqlite release pins.
Full Changelog: v0.5.0...v0.6.0
v0.5.0
Highlights
- Loadable Go extensions — 17 sub-packages under
ext/, each independent, each with aRegister(*Conn) errorentry plus a blank-import<name>/autovariant. - Four new VFS sub-packages —
vfs/crypto(encryption at rest),vfs/cksm(page checksum trailer),vfs/mvcc(in-memory snapshot isolation),vfs/memdb(plain in-memory). Plusvfs.NewReaderforio.ReaderAt-backed databases. - Incremental BLOB API —
(*Conn).OpenBlobreturns a*Blobthat implementsio.ReaderAt/io.WriterAtwithReopenfor rebinding without realloc. - Modern Config short-hands —
sqlite.OpenInMemory(),OpenWAL(path),OpenReadOnly(path),OpenShared(name)plus matching gorm-side helpers. Typed enums (JournalMode,Synchronous,TempStore,CacheMode) replace stringly-typed Pragma values. - VFS chaining —
vfs/cryptoandvfs/cksmacceptOptions.WrapVFS, enabling the checksum-then-encrypt stack with one open database.
New features
ext/ catalog
Scalars / aggregates / collations: regexp, uuid, hash, ipaddr, zorder, stats, unicode. Virtual tables: array, csv, lines, statement, closure, pivot. Specialised stores: bloom (persistent Bloom filter), spellfix1 (fuzzy match). I/O: blobio, fileio. Plus ext/regexp/gorm exposing WhereRegex for GLOB-prefix + REGEXP LIKE-optimization on gorm queries.
Root-package additions
(*Conn).OpenBlob(schema, table, column, rowid, write)→*Blob(*Conn).EnableChecksums(schema)— one-call cksm activation (reserved_bytes=8+ VACUUM)(*Conn).IsInterrupted()— poll the SQLite interrupt flag for vtab Go-side loops(*Conn).StmtCacheStats()→{Hits, Misses, Evictions}counters(*Conn).FileControlReserveBytes(dbName, n)—SQLITE_FCNTL_RESERVE_BYTESwrapper(*Conn).CreateModule,CreateEponymousModule,DeclareVTab— Go-implemented vtab modules withxCreate/xConnectsplit for persistent shadow tablessqlite.RegisterAutoHook(reg)— chains a Register hook onto the default driver'sConnectHook, preserving prior. Underpins theext/<name>/autopattern.sqlite.InMemoryconstant +OpenInMemory/OpenWAL/OpenReadOnly/OpenSharedsqlite.Pointernow recognisestime.Timeas a primitive (no Pointer wrap)stmt.ColumnCount/ColumnName/ColumnDeclType/BindCount/BindName
Hybrid search
fusion.RRFandRRF2— Reciprocal Rank Fusion withWithK/WithWeights/WithLimitoptions.
VFS surface
vfs/crypto: Adiantum (32-byte key, default) or AES-XTS-256 (64-byte key) withOptions.Cipher; per-IORecorderobservability;crypto.DeriveKey(passphrase, salt, cipher)for Argon2id derivation.vfs/cksm: 8-byte Fletcher trailer per page; on-disk compatible with SQLite'scksumvfs.vfs/mvcc: snapshot-isolated reads + atomic-publish writes; shared (file:/name) vs private (file:name) DBs.vfs/memdb: smaller-surface in-memory VFS — direct per-page store undersync.RWMutex, no snapshot copy on commit.vfs.NewReader(io.ReaderAt, size)— wraps a backing reader as a one-file read-only VFS.
Improvements
Performance
fts.Indexpre-renders Insert/Delete SQL once atNew/Open(skip per-rowfmt.Sprintf+strings.Repeat).fts.assignSQLTypefast-paths skip[]byte(v)/string(v)copies on matching T (hot Search path).fts.SearchSliceandvec.KNNSlicepre-size from parsedWithLimit, clamped to[0, 1024].pointerRegistrymutex →RWMutex— concurrentloadPointerno longer serialises on UDF hot paths.vec.Tablepre-rendersinsertSQL/updateSQL/deleteSQLatCreate/Open.
Ergonomics
internal/testhelp.OpenPinned/RawConn/RegisterOn/WithConnectHookas the canonical pinned-conn fixture set.sqlite.DriverName/DriverNameMattnconstants.convert.goerror wrapping switched from%vto%w(errors.Is/errors.Asfriendly).SQLiteContexttype alias =FunctionContextso mattn UDF callback signatures compile after import-path swap.
Observability
(*Conn).StmtCacheStats()per-conn cache telemetry.docs/perf.mdbaseline numbers for Pointer bind, stats aggregates, CSV scan, Bloom membership.
Bug fixes
- Hook handler maps drained on
(*conn).Close— previously leaked closures +*libc.TLSfor the process lifetime; a stale callback could fire on a recycleduintptrhandle. - UDF / collation / aggregate ids reclaimed on Close — same drain pattern, applied to
xFuncs/xCollations/xAggregateFactories. Deserializeno longer leakspBufon a non-OKXsqlite3_deserializeerror.bindText/bindBlobreject payloads >math.MaxInt32instead of truncating to a negative length (whichsqlite3_bind_*interpreted as "usestrlen" → silent corruption).OpenBlob/Reopenrefuse blobs with negativeint32size (a >2 GiB blob would sign-extend to negativeint64and break every bounds check).(*Backup).CommitandFinishare idempotent across both orders — Commit zeros the handle so a follow-up Finish/Close is a no-op rather than a doublesqlite3_backup_finish.fts.Insert/fts.Delete/vec.BatchInsertuseerrors.Join(workErr, tx.Rollback())so a rollback failure doesn't disappear behind the original error.ext/pivotcursor'sCloseno longer leaks the scan stmt'spsqlCString perFiltercall.vfs/cryptoandvfs/cksmNewerror paths now freecvfs+cname+tlswheninitIoMethodsfails (previously leaked per retry under degraded memory).vfs/mvccxClosedrops any heldwriteMuon abnormal close paths (context cancel mid-tx) to prevent next-writer deadlock on shared DBs.ext/blobioopenblob()restricts its write flag to{0, 1}(the previousv != 0silently treated typo-1as "open for write").ext/csvuintArgwidened from 15 bits (max 32767) to 31 (math.MaxInt32).ext/spellfix1createCtorrejects module arguments instead of silently dropping them.ext/csvempty source —header=onerrors clearly;header=offdeclares a single TEXT column and SELECT yields zero rows.vec.Openuses fullValidIdentadmission instead of empty-string check (closes string-builder injection).vec.KNNSliceandObservable.KNNSliceclamp negativeksomake([]T, 0, k)doesn't panic.gorm/migrator.ColumnTypesdeferredrows.Closeno longer clobbers a real failure with a happy Close return.- In-memory connection survives
sqlite3_interrupt— port of modernc.org/sqlite #196. A cancelledQueryContextno longer destroys the entire:memory:database. vec/gormandfts/gormsoft-delete discovered by type (reflect.TypeFor[gorm.DeletedAt]()) instead of field name, soRemovedAt gorm.DeletedAt/ArchivedAt gorm.DeletedAtparticipate in sidecar sync.
Breaking changes
crypto.DeriveKeynow returns([]byte, error)instead of[]byte; a too-short salt becomes a returned error instead of a panic. Migration: addif err != nilhandling at every call site (the three bundled examples show the shape).fusion.RRFandfusion.RRF2return([]Result[K], error)instead of[]Result[K]; mismatchedWithWeightslength returns an error instead of panicking. Migration: capture the error return.unicode.RegisterLikepackage-level bool removed. Migration: passunicode.WithLike()as an option tounicode.Register(c, unicode.WithLike()).cksm.EnableChecksums(c, schema)free function removed. Migration: use(*sqlite.Conn).EnableChecksums(schema)on the connection directly.fts.Column(name, q)no longer panics on a name that failsValidIdent. FTS5 will reject the malformed query at exec time instead. Migration: validate column names withfts.ValidIdentat the API boundary if they come from untrusted input.
Internal / dev
internal/cabiconsolidates the Go↔C function-pointer dance:FuncPointer,AsFunc,Registry[T],PtrMap[T],UniqueName, plus theCallX*family forTsqlite3_io_methodsslot dispatch.internal/sqlidconsolidates SQL-identifier toolkit:NamedArg,Unquote,QuoteIdent,QuoteIdentBacktick,ValidIdent,ToNamedValues,IsAlreadyExistsErr,AsString,AsInt64,AsFloat.internal/gormbridgeconsolidates reflect/gorm plumbing shared byvec/gormandfts/gorm(IndirectType,FindDeletedAtField,IterateRows,MaterializeByRowid[T],ActivePool, etc.).internal/obsconsolidates slog level-dispatch for thevecandftsObservablewrappers.internal/raceskipandinternal/testhelplifted from per-package duplicates.ext/internal/filevtabconsolidates the file-backed-vtab scaffolding shared byext/csvandext/lines(UTF8BOM,OSFS,OpenSource,FullScanBestIndex).vfs/internal/memioconsolidates page-overlap copy betweenvfs/memdbandvfs/mvcc.- CI matrix tests both supported Go releases.
compat_mattn.gorenamed tocompat_sqlite3.go; aliases preserved.- Composite
actions/setupfor shared CI Go setup.
Dependencies
modernc.org/sqlitebumped (carries the SQLite engine pin);modernc.org/libcrides along.golang.org/x/crypto,golang.org/x/text,golang.org/x/syscarried bygo mod tidy.github.com/google/uuidandgolang.org/x/textmoved from indirect to direct asext/uuidandext/unicodeimport them at the leaf.
Acknowledgements
ncruces/go-sqlite3 credit added to LICENSE.ncruces + README. Several ext/ sub-packages are Go-native ports of upstream loadable extensions — re-implemented against this driver's (*Conn).RegisterFunc / vtab helper APIs rather than copied verbatim, with a credit header in each ported package's doc.go.
**Full Chan...
v0.4.0
Highlights
vfs/crypto— pure-Go encryption at rest. Page-level encryption of main DB + journal + WAL + temp files via Adiantum or AES-XTS-256. No CGo, no plaintext fallthrough on disk. Domain-separated tweak per file kind blocks cross-file ciphertext substitution.sqlite.Config— modern Go-typed open shared bydatabase/sqland gorm.sqlite.Open(Config)returns*sqlite.DB;sqlitegorm.OpenConfig(Config)returns*sqlitegorm.DB. One struct, no DSN string assembly, encryption inline, singledefer db.Close()for both pool and VFS lifecycle.fusion— Reciprocal Rank Fusion (Cormack 2009) for combiningvec.KNNandfts.Searchrankings into a hybrid-search result list. Includes aRRF2convenience for the two-slice case.- Custom window functions —
(*Conn).RegisterWindowFunctionexposes Go-implemented window aggregates with optionalWindowFinalizer. MirrorsRegisterAggregatorshape. - FTS5 external-content sync triggers —
fts.SyncTriggersmaterializes the AFTER INSERT / UPDATE / DELETE triggers so external-content indexes stay in step with the source table without manual SQL. - Multi-field vec KNN + bridge SQL hatches —
vecgorm.WithFieldselects a specific embedding column on a model with several;vec.KNNSQL/fts.SearchSQL(and gorm-bridge versions) return raw SQL + bindings for callers wiring the typed result into broader queries.
New packages
github.com/go-again/sqlite/vfs/crypto
Pure-Go encryption-at-rest VFS. Transparent page-level cipher of every SQLite file (main DB, journal, WAL, temp). Two cipher choices:
Adiantum(default) — 32-byte key, tweakable wide-block, fast on every platform.AESXTS— 64-byte key, AES-XTS-256, the disk-encryption standard.
crypto.New(Options{Key, Cipher, PageSize, Recorder}) registers a per-instance VFS the driver routes through; crypto.DeriveKey(passphrase, salt, cipher) runs Argon2id with cipher-sized output. Recorder exposes per-IO observability (slog-shaped helper included).
github.com/go-again/sqlite/fusion
Rank fusion helpers for hybrid search. Pure Go, no SQLite dependency:
RRF[K](rankings ...[]K, opts ...Option) []Hit[K]RRF2[K](a, b []K, opts ...Option) []Hit[K]— two-slice convenience.
Deterministic tiebreak; tunable k constant. Designed to compose vec.KNN + fts.Search keys into a single ordered hybrid result.
Added
Modern Config open
sqlite.Configstruct:Path,Mode,Pragmas,Encryption,VFS,MaxOpenConns,MaxIdleConns,ConnMaxLifetime.sqlite.Pragmas:JournalMode,BusyTimeout,Synchronous,ForeignKeys,CacheSize,TempStore,Extra.sqlite.RecommendedPragmas()returns the production preset (WAL + busy_timeout=5s + foreign_keys).sqlite.Open(Config) (*DB, error)— pool + VFS lifecycle in one wrapper.(*DB).VFSName(),(*DB).Close()— idempotent close, bundles encryption VFS unregistration.sqlite.BuildDSN(Config) string— render the full DSN (including PRAGMAs) for legacy-DSN integrations.sqlite.ApplyPragmas(*sql.DB, Pragmas) error— apply typed PRAGMAs to a pool opened via baresql.Open(dsn).sqlite.Cipher/sqlite.Adiantum/sqlite.AESXTSre-exported so root consumers don't need a separate crypto import.sqlitegorm.OpenConfig(sqlite.Config, ...*gorm.Config) (*DB, error)— same Config, gorm-flavored return.
PRAGMAs ride in as DSN _pragma= URL flags, so per-connection settings (busy_timeout, foreign_keys, synchronous, cache_size, temp_store) propagate to every conn in the pool, not just the one database/sql happens to dispatch the first Exec to.
Vector search
vec.Encode(embeddings)helper for the binary float32 format.(*Table).KNNacceptsWithFilter(renamed fromWithWhere) andWithFieldfor multi-field models.vec.KNNSQL[T]returns raw SQL + bindings for callers stitching KNN into broader queries.- Idempotent
(*Table).Create— re-running CREATE on an existing matching schema is a no-op. vecgorm.KNNSQL[T]/WithFieldmirror the typed surface at the gorm bridge.vecgorm.Embeddingwrapper type so models can declareEmbedding vecgorm.Embedding \vec:"dim=384;metric=cosine"`without a siblinggorm:"-"` tag.
Full-text search
fts.WithFilterquery option for column-aware filtered search.fts.SyncTriggersmaterializes external-content AFTER INSERT / UPDATE / DELETE triggers.fts.ColumnwithUNINDEXEDmodifier for content columns excluded from the FTS5 inverted index but kept for snippet / highlight retrieval.fts.SearchSQL[K, V]raw-SQL escape hatch.- Idempotent
(*Index).Create. ftsgorm.SearchSQL[T]at the gorm bridge.fts.Hit[K, V]is now the canonical result type (renamed fromfts.Match; old name kept as a deprecated alias for one release).
gorm bridges
Migrator.DropTablecascades through any plugin implementingDropTableHookso vec/fts sidecars are torn down with their parent in one call.- vec/gorm and fts/gorm
Plugin()registers the cascade hook automatically. - gorm.Transaction interaction with sidecar writes is now safe: sidecar inserts ride the outer transaction instead of opening a private one.
Custom window functions
(*Conn).RegisterWindowFunction(name string, fn func() WindowAccumulator, deterministic bool) error.- Optional
WindowFinalizerinterface for accumulators that need a finalize step distinct fromValue.
Examples
examples/sqlite-config—sqlite.Open(Config)plain + encrypted.examples/gorm-config—sqlitegorm.OpenConfigplain + encrypted with an FK relation.examples/gorm-crypto— comprehensive stack: gorm + vec/gorm + fts/gorm + fusion + crypto + slog Recorder.examples/vfs-crypto— bare encryption-at-rest demo.examples/window-function—RegisterWindowFunctionround-trip.examples/fusion-hybrid-search— vec KNN + fts Search combined viafusion.RRF2.
Changed
vec.WithWhererenamed tovec.WithFilter; old name removed (no external consumers as of v0.3).vec.Match/vec.Itemconsolidated tovec.Neighbor/vec.Rowfor naming consistency with the gorm bridge.fts.Matchrenamed tofts.Hit; deprecated alias kept for one release.vec/encoding.goreuses scan targets across rows inFTS5 Searchfor fewer allocations on large result sets.vecKNNkpre-allocation now capped to avoid OOM on adversarial inputs.- vfs/crypto io-methods table bumped to
iVersion=2withxShmtrampolines forwarding to the default unix VFS, soPRAGMA journal_mode = WALnow succeeds through the encryption path (previously fell back toDELETEsilently).
Fixed
vfs/crypto: cross-file ciphertext substitution blocked via 1-byte file-kind tag in the tweak (Adiantum) / high 8 bits of the XTS sector index.vfs/crypto: Adiantum concurrency race on the sharedhashBuf— every cipher op now holds a mutex matching the upstream library's documented single-threaded contract.vfs/crypto: defensive key copy innewCipherso callers can zero or mutateOptions.KeyafterNewreturns.gorm:DATETIMEcolumns scanned intoany/mapnow returntime.Time(matches mattn behavior; previously returned RFC3339 strings).vec/gorm: nestedgorm.Transactionrolls back sidecar writes correctly.vec/gorm:Migrator.DropTablecascades sidecar cleanup so orphans no longer accumulate acrossAutoMigratecycles.fts/gorm: external-content triggers stay aligned afterAlterColumn/recreateTable.
Compatibility
- Backward-compatible with every v0.3 entry:
sql.Open("sqlite", dsn),sql.Open("sqlite3", dsn),sqlitegorm.Open(dsn),sqlitegorm.New(Config{DSN}), andcrypto.New(Options)all keep working unchanged. - Supported Go versions: the two most recent releases (1.26 + 1.25).
- Supported platforms: same matrix as modernc.org/sqlite. All 15 darwin / freebsd / linux / windows targets pass
cross-build. - Bundled SQLite: 3.53.1 (from modernc.org/sqlite). libc pin carried automatically via
go mod tidy.
Acknowledgements
vfs/crypto: Adiantum from lukechampine.com/adiantum; AES-XTS from golang.org/x/crypto/xts; Argon2id from golang.org/x/crypto/argon2. Threat model and io-methods drift discipline live in vfs/crypto/doc.go.
Full Changelog: v0.3.0...v0.4.0
v0.3.0
Transactions
vec/gorm and fts/gorm callbacks route writes through the active gorm.ConnPool exposed on db.Statement — the live *sql.Tx when inside gorm.Transaction, the *sql.DB otherwise. Sidecar writes commit and roll back with the parent gorm transaction. ftsgorm.Search sees read-your-own-writes inside a tx.
vec/gorm.KNN[T] reads through *sql.DB and is therefore not supported inside a gorm.Transaction (it would deadlock under the typical pinned-conn SQLite fixture). Use tx.Raw against the <table>_vec sidecar when you need to query uncommitted state.
Regression tests cover Create / batch-Create / Update / Delete / soft-delete rollback in vec/gorm, plus in-table / contentless / external-mode rollback and search-sees-uncommitted-writes in fts/gorm.
API
vec.Neighbor— KNN result.vec.Row— input toBatchInsert.fts.Hit[K, V]— search hit returned byIndex.Search/Index.SearchSlice.vecgorm.Hit[T]/ftsgorm.Hit[T]— typed gorm-bridge hit pairing the model with the distance / rank.vec.WithFilter(sql, args...)/vecgorm.WithFilter(sqlFragment, args...)/ftsgorm.WithFilter(sqlFragment, args...)— uniform filter option across all three layers. The fragment is caller-trusted raw SQL (parameters bind viaargs, fragment interpolates as-is — same contract asgorm.Where). Validate identifiers viavec.ValidIdent/fts.ValidIdentbefore interpolating. Each variant carries an explicit# Trust modeldocstring section.vecgorm.ErrNotInstalled/ftsgorm.ErrNotInstalled— sentinel errors returned by helper APIs when the plugin hasn't been installed viadb.Use(...).errors.Isagainst either works.
Performance
fts.Index.Search:scanTargets/holdersslices hoisted above the per-row loop. Drops per-row allocations from two to zero.vec.Table.KNNSlice: caps result-slice pre-allocation atmin(k, 1024). A buggy or malicious caller passing hugekno longer pre-allocates gigabytes before the query runs.registerSchemain both gorm bridges: tightened double-checked locking so concurrent first-access on the same model type only pays theschema.Parsecost once.- Per-row reflection in the bridge callbacks uses a cached field-index chain (parsed once per model type).
Benchmarks
New benchmarks (deterministic PCG-seeded corpora, dim=384, 10k docs):
BenchmarkKNN,BenchmarkKNN_WithFilter,BenchmarkBatchInsert_JSON,BenchmarkBatchInsert_Binaryinvec/.BenchmarkSearch,BenchmarkSearch_WithRanking,BenchmarkInsertinfts/.- Wired into the justfile as
bench-vec/bench-fts.
Concurrency
vec/gorm/concurrent_test.go: 16 goroutines × 8 calls hammering the schema cache undergo test -race. Pins the double-checked locking pattern.
Tooling
just lintchain expanded tofmt-check + vet + staticcheck + golangci-lint + gopls modernize. Localjust lintmirrors CI byte-for-byte.gopls modernizeadded to CI as well. Forked modernc/glebarez files are filtered out; our code is held to the current-Go-syntax bar.- Codebase swept for the modernizations gopls catches:
range-over-int,reflect.TypeFor,strings.SplitSeq,sync.WaitGroup.Go,atomic.Int32,errors.Join.
Docs
- New
llms.txt(llmstxt.org-format index) andllms-full.txt(consumer guide: driver-name picker, DSN flag whitelist, sub-package map, pitfalls, migration recipes, what-NOT-to-do). Aimed at LLM agents in downstream projects. - README: new "Supported Go versions" section codifying the rolling-window policy.
CLAUDE.md"Always" PR checklist enumerates every doc that must move together (doc.go, README, llms.txt, llms-full.txt,docs/coverage-*.md) to prevent doc drift.plan-doc-sweep.mdinventory lists the new llms files and reminds the sweeper to mirror DSN-flag / migration-recipe / pitfall changes intollms-full.txt.
Supported Go versions
Two most recent Go releases only: today Go 1.26 (current) and Go 1.25 (previous). Older Go drops out within one release cycle of each new minor. Modern syntax is treated as a feature — generics, iter.Seq2, log/slog, generic type aliases, range over int, sync.WaitGroup.Go, strings.SplitSeq, reflect.TypeFor are all in use.
Stability
Still pre-v1.0. Public API is settling but not frozen. Pin the version you use in go.mod.
Full Changelog: v0.2.0...v0.3.0
v0.2.0
Highlights
- Tag-driven gorm integration for vec and FTS5. Annotate a field on your gorm model, register the plugin, and the sidecar table or external-content FTS5 index maintains itself — including
CreateInBatchesin a single transaction, soft-delete filtering, andDropTablecascade. - License is now Apache 2.0, up from BSD 3-Clause. Upstream attributions preserved verbatim in
LICENSE.modernc/LICENSE.mattn/LICENSE.glebarezplus aNOTICEfile. Existing usage is unaffected; redistributors should refresh their bundled license files. - Three new CI-enforced upstream-suite lanes. The full
gorm.io/gorm/testsintegration suite, plus vendored subsets ofmodernc.org/sqlite's andmattn/go-sqlite3's own test suites, run against this driver on every push. - SQL conformance suite under
tests/sql/— a methodical, feature-by-feature matrix of the raw SQL surface (SELECT clauses, joins, CTEs, window functions, JSON1, datetime modifiers, every constraint type, triggers, UPSERT, RETURNING, PRAGMAs, ATTACH, VACUUM, generated columns, WITHOUT ROWID, STRICT).
New packages
github.com/go-again/sqlite/vec/gorm
Tag-driven bridge between gorm models and vec.Table sidecars.
type Document struct {
ID uint `gorm:"primaryKey"`
Embedding vecgorm.Embedding `vec:"dim=384;metric=cosine"`
}
db.Use(vecgorm.Plugin())
vecgorm.Migrate(db, &Document{})
db.Create(&doc) // sidecar populated
results, _ := vecgorm.KNN[Document](ctx, db, queryVec, 5) // returns []Result[Document]
db.Migrator().DropTable(&Document{}) // cascades to sidecarSupported tag keys: dim, metric, encoding, table, column. Field type can be vecgorm.Embedding (recommended) or []float32 with gorm:"-". Options: WithFilter(sql, args...), IncludeDeleted().
github.com/go-again/sqlite/fts/gorm
Tag-driven bridge for FTS5 search.
type Article struct {
ID uint `gorm:"primaryKey"`
Title string `fts5:"tokenize=porter+unicode61"`
Body string `fts5:"tokenize=porter+unicode61"`
}
db.Use(ftsgorm.Plugin())
ftsgorm.Migrate(db, &Article{}) // table + triggers
db.Create(&a) // trigger updates FTS5
results, _ := ftsgorm.Search[Article](ctx, db, fts.Term("fox"))Three FTS5 modes selectable via tag: external (default; triggers-driven), external=false (in-table), contentless=true (index only — Snippet/Highlight rejected at search time). Tag keys: tokenize, prefix, column, table, detail, external, contentless. Options: WithLimit, WithOffset, WithRanking, WithSnippet, WithHighlight, IncludeDeleted.
tests/sql/
Standalone SQL conformance test suite organized by SQLite Language Reference category. Runs on linux / macos / windows in CI. See docs/coverage-sql.md.
API additions
vec.Metric.Keyword() string— returns thevec0()constructor keyword (l2/cosine/l1).vec.Encoding.Encode([]float32) anyandvec.Encoding.Placeholder() string— replace the previously-privateencodeValue/matchPlaceholderhelpers; the old names remain as internal aliases for backwards compat.vec.QuoteIdent/vec.ValidIdent/fts.QuoteIdent/fts.ValidIdent— previously private; exported so sub-packages and advanced users can reuse them.fts.Query.Build() string— renamed from the previously-unexportedbuild()so the gorm bridge (and anyone else) can render MATCH expressions.sqlite.gorm.DropTableHookinterface — gorm plugins can implement this to cascade sidecar cleanup whendb.Migrator().DropTable(&Model{})runs.
Bug fixes
- gorm
Table(...).Find(&map)returnstime.Timefor DATETIME columns instead of an RFC3339 string. Fixed by auto-injecting_texttotime=1on DSNs opened through the gorm dialector. Aligns map-mode reads with mattn behavior and fixes the upstreamTestFind/FirstMapWithTable/Birthdayassertion. - gorm
Migrator.DropTablenow cascades. Callingdb.Migrator().DropTable(&Model{})cleans up the vec0 sidecar / FTS5 table + triggers without an explicit second call. Implemented via the newDropTableHookinterface. - CI gorm-upstream lane now reports correct PASS/FAIL counts. GitHub Actions writes per-line timestamps into the tee'd log, which broke our anchored
grep -cE '^--- PASS:'and aborted the script withpipefail+errexit. Drop the^anchor, capture go test's real exit via${PIPESTATUS[0]}, survive zero-match grep with|| true.
Compatibility notes
- License: BSD-style → Apache 2.0. NOTICE attributions added for modernc, mattn, glebarez, sqlite-vec, and SQLite. Adopters redistributing this package should refresh their bundled license disclosures.
- vec-tagged gorm fields must use either
vecgorm.EmbeddingOR[]float32withgorm:"-". The plugin emits a clear error if both escape hatches are missing — gorm'sschema.Parserejects bare[]float32before any plugin hook can intervene. - Existing direct users of
vec.Table/fts.Index/ the gorm dialector are unaffected by the bridge packages — they're additive.
Internals
- Removed
metricKeyword/encodeValue/matchPlaceholder/quote/validIdentduplication acrossvec,fts,vec/gorm,fts/gorm. Single source of truth on thevec.Metric/vec.Encodingmethods and the exportedQuoteIdent/ValidIdenthelpers. - Per-file copyright headers stripped from authored files; modernc-derived files keep their upstream copyright lines as required by BSD 3-Clause preservation.
Documentation
docs/coverage-sql.md— methodical SQL surface matrix.docs/coverage-gorm.md— "Deep integration" section covering both bridges' tag syntax, lifecycle, and supported modes.docs/gorm-upstream.md/docs/modernc-upstream.md/docs/mattn-upstream.md— reproduction recipes for the three CI-enforced upstream suites.examples/gorm-vec-tagged/andexamples/gorm-fts-tagged/— runnable end-to-end demos of the tag-driven flow.- README gained "Deep gorm integration" + "Sponsors" sections.
Full Changelog: v0.1.1...v0.2.0
v0.1.0
Full Changelog: https://github.com/go-again/sqlite/commits/v0.1.0