Skip to content

build!: bump tree-sitter core v0.25.1 → v0.26.6 #51

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
DeusData wants to merge 1 commit into
base: master
Choose a base branch
from
Open

build!: bump tree-sitter core v0.25.1 → v0.26.6 #51

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 48 additions & 82 deletions include/tree_sitter/api.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ typedef uint16_t TSStateId;
typedef uint16_t TSSymbol;
typedef uint16_t TSFieldId;
typedef struct TSLanguage TSLanguage;
typedef struct TSLanguageMetadata TSLanguageMetadata;
typedef struct TSParser TSParser;
typedef struct TSTree TSTree;
typedef struct TSQuery TSQuery;
Expand All @@ -52,12 +51,15 @@ typedef struct TSLookaheadIterator TSLookaheadIterator;
// This function signature reads one code point from the given string,
// returning the number of bytes consumed. It should write the code point
// to the `code_point` pointer, or write -1 if the input is invalid.
typedef uint32_t (*DecodeFunction)(
typedef uint32_t (*TSDecodeFunction)(
const uint8_t *string,
uint32_t length,
int32_t *code_point
);

// Deprecated alias to be removed in ABI 16
typedef TSDecodeFunction DecodeFunction;

typedef enum TSInputEncoding {
TSInputEncodingUTF8,
TSInputEncodingUTF16LE,
Expand Down Expand Up @@ -88,7 +90,7 @@ typedef struct TSInput {
void *payload;
const char *(*read)(void *payload, uint32_t byte_index, TSPoint position, uint32_t *bytes_read);
TSInputEncoding encoding;
DecodeFunction decode;
TSDecodeFunction decode;
} TSInput;

typedef struct TSParseState {
Expand Down Expand Up @@ -297,16 +299,7 @@ const TSRange *ts_parser_included_ranges(
* are four possible reasons for failure:
* 1. The parser does not have a language assigned. Check for this using the
[`ts_parser_language`] function.
* 2. Parsing was cancelled due to a timeout that was set by an earlier call to
* the [`ts_parser_set_timeout_micros`] function. You can resume parsing from
* where the parser left out by calling [`ts_parser_parse`] again with the
* same arguments. Or you can start parsing from scratch by first calling
* [`ts_parser_reset`].
* 3. Parsing was cancelled using a cancellation flag that was set by an
* earlier call to [`ts_parser_set_cancellation_flag`]. You can resume parsing
* from where the parser left out by calling [`ts_parser_parse`] again with
* the same arguments.
* 4. Parsing was cancelled due to the progress callback returning true. This callback
* 2. Parsing was cancelled due to the progress callback returning true. This callback
* is passed in [`ts_parser_parse_with_options`] inside the [`TSParseOptions`] struct.
*
* [`read`]: TSInput::read
Expand Down Expand Up @@ -364,50 +357,14 @@ TSTree *ts_parser_parse_string_encoding(
/**
* Instruct the parser to start the next parse from the beginning.
*
* If the parser previously failed because of a timeout or a cancellation, then
* If the parser previously failed because of the progress callback, then
* by default, it will resume where it left off on the next call to
* [`ts_parser_parse`] or other parsing functions. If you don't want to resume,
* and instead intend to use this parser to parse some other document, you must
* call [`ts_parser_reset`] first.
*/
void ts_parser_reset(TSParser *self);

/**
* @deprecated use [`ts_parser_parse_with_options`] and pass in a callback instead, this will be removed in 0.26.
*
* Set the maximum duration in microseconds that parsing should be allowed to
* take before halting.
*
* If parsing takes longer than this, it will halt early, returning NULL.
* See [`ts_parser_parse`] for more information.
*/
void ts_parser_set_timeout_micros(TSParser *self, uint64_t timeout_micros);

/**
* @deprecated use [`ts_parser_parse_with_options`] and pass in a callback instead, this will be removed in 0.26.
*
* Get the duration in microseconds that parsing is allowed to take.
*/
uint64_t ts_parser_timeout_micros(const TSParser *self);

/**
* @deprecated use [`ts_parser_parse_with_options`] and pass in a callback instead, this will be removed in 0.26.
*
* Set the parser's current cancellation flag pointer.
*
* If a non-null pointer is assigned, then the parser will periodically read
* from this pointer during parsing. If it reads a non-zero value, it will
* halt early, returning NULL. See [`ts_parser_parse`] for more information.
*/
void ts_parser_set_cancellation_flag(TSParser *self, const size_t *flag);

/**
* @deprecated use [`ts_parser_parse_with_options`] and pass in a callback instead, this will be removed in 0.26.
*
* Get the parser's current cancellation flag pointer.
*/
const size_t *ts_parser_cancellation_flag(const TSParser *self);

/**
* Set the logger that a parser should use during parsing.
*
Expand Down Expand Up @@ -751,6 +708,24 @@ void ts_node_edit(TSNode *self, const TSInputEdit *edit);
*/
bool ts_node_eq(TSNode self, TSNode other);

/**
* Edit a point to keep it in-sync with source code that has been edited.
*
* This function updates a single point's byte offset and row/column position
* based on an edit operation. This is useful for editing points without
* requiring a tree or node instance.
*/
void ts_point_edit(TSPoint *point, uint32_t *point_byte, const TSInputEdit *edit);

/**
* Edit a range to keep it in-sync with source code that has been edited.
*
* This function updates a range's start and end positions based on an edit
* operation. This is useful for editing ranges without requiring a tree
* or node instance.
*/
void ts_range_edit(TSRange *range, const TSInputEdit *edit);

/************************/
/* Section - TreeCursor */
/************************/
Expand Down Expand Up @@ -1092,26 +1067,6 @@ bool ts_query_cursor_did_exceed_match_limit(const TSQueryCursor *self);
uint32_t ts_query_cursor_match_limit(const TSQueryCursor *self);
void ts_query_cursor_set_match_limit(TSQueryCursor *self, uint32_t limit);

/**
* @deprecated use [`ts_query_cursor_exec_with_options`] and pass in a callback instead, this will be removed in 0.26.
*
* Set the maximum duration in microseconds that query execution should be allowed to
* take before halting.
*
* If query execution takes longer than this, it will halt early, returning NULL.
* See [`ts_query_cursor_next_match`] or [`ts_query_cursor_next_capture`] for more information.
*/
void ts_query_cursor_set_timeout_micros(TSQueryCursor *self, uint64_t timeout_micros);

/**
* @deprecated use [`ts_query_cursor_exec_with_options`] and pass in a callback instead, this will be removed in 0.26.
*
* Get the duration in microseconds that query execution is allowed to take.
*
* This is set via [`ts_query_cursor_set_timeout_micros`].
*/
uint64_t ts_query_cursor_timeout_micros(const TSQueryCursor *self);

/**
* Set the range of bytes in which the query will be executed.
*
Expand Down Expand Up @@ -1146,6 +1101,28 @@ bool ts_query_cursor_set_byte_range(TSQueryCursor *self, uint32_t start_byte, ui
*/
bool ts_query_cursor_set_point_range(TSQueryCursor *self, TSPoint start_point, TSPoint end_point);

/**
* Set the byte range within which all matches must be fully contained.
*
* Set the range of bytes in which matches will be searched for. In contrast to
* `ts_query_cursor_set_byte_range`, this will restrict the query cursor to only return
* matches where _all_ nodes are _fully_ contained within the given range. Both functions
* can be used together, e.g. to search for any matches that intersect line 5000, as
* long as they are fully contained within lines 4500-5500
*/
bool ts_query_cursor_set_containing_byte_range(TSQueryCursor *self, uint32_t start_byte, uint32_t end_byte);

/**
* Set the point range within which all matches must be fully contained.
*
* Set the range of bytes in which matches will be searched for. In contrast to
* `ts_query_cursor_set_point_range`, this will restrict the query cursor to only return
* matches where _all_ nodes are _fully_ contained within the given range. Both functions
* can be used together, e.g. to search for any matches that intersect line 5000, as
* long as they are fully contained within lines 4500-5500
*/
bool ts_query_cursor_set_containing_point_range(TSQueryCursor *self, TSPoint start_point, TSPoint end_point);

/**
* Advance to the next match of the currently running query.
*
Expand Down Expand Up @@ -1262,17 +1239,6 @@ const char *ts_language_symbol_name(const TSLanguage *self, TSSymbol symbol);
*/
TSSymbolType ts_language_symbol_type(const TSLanguage *self, TSSymbol symbol);

/**
* @deprecated use [`ts_language_abi_version`] instead, this will be removed in 0.26.
*
* Get the ABI version number for this language. This version number is used
* to ensure that languages were generated by a compatible version of
* Tree-sitter.
*
* See also [`ts_parser_set_language`].
*/
uint32_t ts_language_version(const TSLanguage *self);

/**
* Get the ABI version number for this language. This version number is used
* to ensure that languages were generated by a compatible version of
Expand Down Expand Up @@ -1417,7 +1383,7 @@ const TSLanguage *ts_wasm_store_load_language(
);

/**
* Get the number of languages instantiated in the given wasm store.
* Get the number of languages instantiated in the given Wasm store.
*/
size_t ts_wasm_store_language_count(const TSWasmStore *);

Expand Down
8 changes: 0 additions & 8 deletions language.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,14 +43,6 @@ func NewLanguage(ptr unsafe.Pointer) *Language {
return &Language{Inner: (*C.TSLanguage)(ptr)}
}

// Deprecated: Use [Language.AbiVersion] instead.
//
// Get the ABI version number that indicates which version of the
// Tree-sitter CLI that was used to generate this [Language].
func (l *Language) Version() uint32 {
return uint32(C.ts_language_version(l.Inner))
}

// Get the ABI version number that indicates which version of the
// Tree-sitter CLI that was used to generate this [Language].
func (l *Language) AbiVersion() uint32 {
Expand Down
83 changes: 17 additions & 66 deletions parser.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ import "C"
import (
"context"
"os"
"sync/atomic"
"unsafe"

"github.com/mattn/go-pointer"
Expand Down Expand Up @@ -178,34 +177,20 @@ func (p *Parser) Parse(text []byte, oldTree *Tree) *Tree {
}, oldTree, nil)
}

// Deprecated: Use [Parser.ParseWithOptions] instead, and handle cancellation in the callback, this will be removed in 0.26.
//
// Parse a slice of UTF8 text.
//
// # Arguments:
// - `ctx` The context to parse with.
// - `text` The UTF8-encoded text to parse.
// - `old_tree` A previous syntax tree parsed from the same document. If the text of the
// document has changed since `old_tree` was created, then you must edit `old_tree` to match
// the new text using [Tree.Edit].
// Deprecated: Use [Parser.ParseWithOptions] instead, and handle
// cancellation in the callback.
func (p *Parser) ParseCtx(ctx context.Context, text []byte, oldTree *Tree) *Tree {
finish := make(chan struct{})

if ctx.Done() != nil {
go func() {
select {
case <-ctx.Done():
atomic.StoreUintptr(p.CancellationFlag(), 1)
case <-finish:
return
}
}()
}

tree := p.Parse(text, oldTree)
close(finish)

return tree
length := len(text)
return p.ParseWithOptions(func(i int, _ Point) []byte {
if i < length {
return text[i:]
}
return []byte{}
}, oldTree, &ParseOptions{
ProgressCallback: func(_ ParseState) bool {
return ctx.Err() != nil
},
})
}

// Deprecated: Use [Parser.ParseUTF16LE] or [Parser.ParseUTF16BE] instead.
Expand Down Expand Up @@ -349,6 +334,7 @@ func (p *Parser) ParseWithOptions(callback func(int, Point) []byte, oldTree *Tre
progress_callback: (*[0]byte)(C.parserProgressCallback),
payload: pointer.Save(options),
}
defer pointer.Unref(cOptions.payload)
}

cNewTree := C.ts_parser_parse_with_options(p._inner, cOldTree, cInput, cOptions)
Expand Down Expand Up @@ -476,6 +462,7 @@ func (p *Parser) ParseUTF16LEWithOptions(callback func(int, Point) []uint16, old
progress_callback: (*[0]byte)(C.parserProgressCallback),
payload: pointer.Save(options),
}
defer pointer.Unref(cOptions.payload)
}

cNewTree := C.ts_parser_parse_with_options(p._inner, cOldTree, cInput, cOptions)
Expand Down Expand Up @@ -547,6 +534,7 @@ func (p *Parser) ParseUTF16BEWithOptions(callback func(int, Point) []uint16, old
progress_callback: (*[0]byte)(C.parserProgressCallback),
payload: pointer.Save(options),
}
defer pointer.Unref(cOptions.payload)
}

cNewTree := C.ts_parser_parse_with_options(p._inner, cOldTree, cInput, cOptions)
Expand Down Expand Up @@ -630,6 +618,7 @@ func (p *Parser) ParseCustomEncoding(
progress_callback: (*[0]byte)(C.parserProgressCallback),
payload: pointer.Save(options),
}
defer pointer.Unref(cOptions.payload)
}

cNewTree := C.ts_parser_parse_with_options(p._inner, cOldTree, cInput, cOptions)
Expand All @@ -652,26 +641,6 @@ func (p *Parser) Reset() {
C.ts_parser_reset(p._inner)
}

// Deprecated: Use [Parser.ParseWithOptions] and pass in a callback instead, this will be removed in 0.26.
//
// Get the duration in microseconds that parsing is allowed to take.
//
// This is set via [Parser.SetTimeoutMicros].
func (p *Parser) TimeoutMicros() uint64 {
return uint64(C.ts_parser_timeout_micros(p._inner))
}

// Deprecated: Use [Parser.ParseWithOptions] and pass in a callback instead, this will be removed in 0.26.
//
// Set the maximum duration in microseconds that parsing should be allowed
// to take before halting.
//
// If parsing takes longer than this, it will halt early, returning `nil`.
// See [Parser.Parse] for more information.
func (p *Parser) SetTimeoutMicros(timeoutMicros uint64) {
C.ts_parser_set_timeout_micros(p._inner, C.uint64_t(timeoutMicros))
}

// Get the ranges of text that the parser will include when parsing.
func (p *Parser) IncludedRanges() []Range {
var count C.uint
Expand Down Expand Up @@ -734,21 +703,3 @@ func (p *Parser) SetIncludedRanges(ranges []Range) error {
return &IncludedRangesError{0}
}

// Deprecated: Use [Parser.ParseWithOptions] and pass in a callback instead, this will be removed in 0.26.
//
// Get the parser's current cancellation flag pointer.
func (p *Parser) CancellationFlag() *uintptr {
return (*uintptr)(unsafe.Pointer(C.ts_parser_cancellation_flag(p._inner)))
}

// Deprecated: Use [Parser.ParseWithOptions] and pass in a callback instead, this will be removed in 0.26.
//
// Set the parser's current cancellation flag pointer.
//
// If a pointer is assigned, then the parser will periodically read from
// this pointer during parsing. If it reads a non-zero value, it will halt
// early, returning `nil`. See [Parser.Parse] for more
// information.
func (p *Parser) SetCancellationFlag(flag *uintptr) {
C.ts_parser_set_cancellation_flag(p._inner, (*C.size_t)(unsafe.Pointer(flag)))
}
2 changes: 1 addition & 1 deletion parser_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -549,7 +549,7 @@ func TestParsingAfterDetectingErrorInTheMiddleOfStringToken(t *testing.T) {
func TestParsingOnMultipleThreads(t *testing.T) {
// Parse this source file so that each thread has a non-trivial amount of
// work to do.
thisFileSource, err := os.ReadFile("tree-sitter/cli/src/tests/parser_test.rs")
thisFileSource, err := os.ReadFile("tree-sitter/crates/cli/src/tests/parser_test.rs")
assert.Nil(t, err)

parser := NewParser()
Expand Down
15 changes: 0 additions & 15 deletions query.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -719,21 +719,6 @@ func (qc *QueryCursor) SetMatchLimit(limit uint) {
C.ts_query_cursor_set_match_limit(qc._inner, C.uint32_t(limit))
}

// Set the maximum duration in microseconds that query execution should be allowed to
// take before halting.
//
// If query execution takes longer than this, it will halt early, returning None.
func (qc *QueryCursor) SetTimeoutMicros(timeoutMicros uint64) {
C.ts_query_cursor_set_timeout_micros(qc._inner, C.uint64_t(timeoutMicros))
}

// Get the duration in microseconds that query execution is allowed to take.
//
// This is set via [QueryCursor.SetTimeoutMicros]
func (qc *QueryCursor) TimeoutMicros() uint64 {
return uint64(C.ts_query_cursor_timeout_micros(qc._inner))
}

// Check if, on its last execution, this cursor exceeded its maximum number
// of in-progress matches.
func (qc *QueryCursor) DidExceedMatchLimit() bool {
Expand Down
Loading