Closed
Conversation
…e#60432) Cherry-picked from apache#59755 Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
### What problem does this PR solve? apache#58636 apache#59835 apache#59830 ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
…query internal table without specifying catalog condition, this incompatible with show partitions command apache#60247 (apache#60452) Cherry-picked from apache#60247 Co-authored-by: heguanhui <hgh_wy163mail@163.com>
… show backends command apache#60210 (apache#60453) Cherry-picked from apache#60210 Co-authored-by: heguanhui <hgh_wy163mail@163.com>
…cache should consider sortByGroupKey apache#60431 (apache#60457) Cherry-picked from apache#60431 Co-authored-by: 924060929 <lanhuajian@selectdb.com>
apache#60464) Cherry-picked from apache#60422 Co-authored-by: Siyang Tang <tangsiyang@selectdb.com>
apache#60404 (apache#60461) Cherry-picked from apache#60404 Co-authored-by: lihangyu <lihangyu@selectdb.com>
Cherry-picked from apache#60392 Co-authored-by: TengJianPing <tengjianping@selectdb.com>
…loud mode apache#60263 (apache#60471) Cherry-picked from apache#60263 Co-authored-by: meiyi <meiyi@selectdb.com>
…che#60181 (apache#60463) Cherry-picked from apache#60181 Co-authored-by: wudi <wudi@selectdb.com>
…pache#59563 (apache#59604) Cherry-picked from apache#59563 Co-authored-by: Luwei <814383175@qq.com>
…ache#60469) ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
…ot alive yet apache#60317 (apache#60470) Cherry-picked from apache#60317 Co-authored-by: deardeng <dengxin@selectdb.com>
…lumns_count` to 2048 apache#60462 (apache#60472) Cherry-picked from apache#60462 Co-authored-by: lihangyu <lihangyu@selectdb.com>
…ator back pressure apache#60291 (apache#60502) Cherry-picked from apache#60291 Co-authored-by: hui lai <laihui@selectdb.com>
…t contains CAST (apache#60485) cherry-pick apache#60448
…g objects (part 2) apache#60286 (apache#60507) Cherry-picked from apache#60286 Co-authored-by: Gavin Chou <gavin@selectdb.com>
…apache#54434 (apache#60509) Cherry-picked from apache#54434 Co-authored-by: Yongqiang YANG <yangyongqiang@selectdb.com>
apache#60514) Cherry-picked from apache#60484 Co-authored-by: Gabriel <liwenqiang@selectdb.com>
…ndex returning wrong results apache#60500 (apache#60516) Cherry-picked from apache#60500 Co-authored-by: Jack <jiangkai@selectdb.com>
…ode for mysql/pg streaming job apache#60473 (apache#60527) Cherry-picked from apache#60473 Co-authored-by: wudi <wudi@selectdb.com>
…ies() does not reset logical properties properly apache#60495 (apache#60521) Cherry-picked from apache#60495 Co-authored-by: minghong <zhouminghong@selectdb.com>
pick apache#59896 This feature adds support for UPDATE_FLEXIBLE_COLUMNS mode in routine load, allowing partial updates where the columns to update are determined dynamically from each JSON row's fields rather than being fixed upfront. Key changes: - Add unique_key_update_mode property to routine load (UPSERT, UPDATE_FIXED_COLUMNS, UPDATE_FLEXIBLE_COLUMNS) - Validate flexible partial update constraints (JSON format only, no jsonpaths, no COLUMNS clause, no fuzzy_parse) - Support ALTER ROUTINE LOAD to change update mode - Add comprehensive regression tests for all scenarios - Add waitForTaskFinishMoW utility for MOW table tests Constraints for flexible partial update: - Only JSON format supported - Cannot use jsonpaths (columns determined from JSON fields) - Cannot use COLUMNS clause - Cannot use fuzzy_parse - Table must be MOW with skip_bitmap column enabled - Table cannot have variant columns ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
… due to wrong finish_time_ apache#60456 (apache#60535) Cherry-picked from apache#60456 Co-authored-by: Wen Zhenghu <wenzhenghu.zju@gmail.com>
…pache#60525 (apache#60562) Cherry-picked from apache#60525 Co-authored-by: 924060929 <lanhuajian@selectdb.com>
apache#60553) …ting benchmark (apache#60139) Uses LOG_DIR if set, otherwise config::sys_log_dir, else a relative pipe_tracing directory. This prevents fmt from seeing a null string pointer. ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> --------- Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
…rows instead of only those in original bitmap apache#60537 (apache#60576) Cherry-picked from apache#60537 Co-authored-by: zzzxl <yangsiyu@selectdb.com>
Cherry-picked from apache#60333 Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
### What problem does this PR solve? Pick apache#60253 apache#60393 apache#60481 apache#56956 apache#60334 apache#60494 ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
…en the rowset is not in `_rowset_warm_up_states` (apache#61238) (apache#61680) pick apache#61238
…ministic for multi-index columns apache#61596 (apache#61692) ### What problem does this PR solve? Issue Number: close #xxx Related PR: apache#61596 Problem Summary: Pick apache#61596 to branch-4.0. When a column has multiple inverted indexes (e.g., different analyzers), select_best_reader could return different readers depending on std::unordered_map iteration order, causing non-deterministic behavior. This fix sorts candidate readers by index_id to ensure deterministic selection. ### Release note None ### Check List (For Author) - Test - [x] Regression test - [x] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [x] No. - [ ] Yes. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label
… remove unused _close_wait apache#61593 (apache#61623) Cherry-picked from apache#61593 Co-authored-by: zclllyybb <zhaochangle@selectdb.com>
…ore filter-join pushdown apache#61635 (apache#61714) Cherry-picked from apache#61635 Co-authored-by: 924060929 <lanhuajian@selectdb.com>
… virtual column apache#61584 (apache#61755) Cherry-picked from apache#61584 Co-authored-by: Jack <jiangkai@selectdb.com>
…TE and DELETE commands apache#61681 (apache#61751) Cherry-picked from apache#61681 Co-authored-by: morrySnow <zhangwenxin@selectdb.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…y cache digest apache#61709 (apache#61718) Cherry-picked from apache#61709 Co-authored-by: 924060929 <lanhuajian@selectdb.com>
…eportExecStatus for non-Load queries apache#61467 (apache#61734) Cherry-picked from apache#61467 Co-authored-by: Mingyu Chen (Rayner) <yunyou@selectdb.com>
…ssion case apache#61720 (apache#61723) Cherry-picked from apache#61720 Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
… 20000 and add near-limit metrics apache#61511 (apache#61764) ## Summary Cherry-pick of apache#61511 to branch-4.0. - Raise `max_dynamic_partition_num` default from 500 to 20000 and `max_auto_partition_num` from 2000 to 20000 to match modern production workloads - Add warning logs when partition counts exceed 80% of their configured limits, enabling proactive detection before hard failures - Add Prometheus counter metrics (`auto_partition_near_limit_count`, `dynamic_partition_near_limit_count`) for monitoring/alerting ## Conflict Resolution - `Config.java`: Trivial context conflict in `max_auto_partition_num` description formatting — resolved by taking the incoming change (20000 default + updated English description). ## Test plan - [ ] Verify existing dynamic partition tests pass with new default - [ ] Verify auto-partition limit check still errors correctly when exceeded - [ ] Verify warning logs appear when partition count is between 80%-100% of limit - [ ] Verify new metrics appear in `/metrics` Prometheus endpoint Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: dataroaring <dataroaring@users.noreply.github.com>
…he#58082) (apache#61193) add check for external data (lakehouse data) when get_tablet_id to avoid out-of-range exception ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
… count and add be UT (apache#61758) (apache#61781) Pick apache#61758 Check multiget result rows matches request row id count when doing merge_multi_response: 1. A BE may return an empty block event if request.request_block_descs(i).row_id_size() != 0: If the id_file_map was GC'd on the BE before it could process the request, refer 'if (!id_file_map)' in RowIdStorageReader::read_by_rowids. 2. Report error in any case where the row count doesn't match, even if it's not empty, since that indicates a bug in BE's row fetching logic or serialization logic. Also add comments. Issue Number: close #xxx Related PR: #xxx Problem Summary: None - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
…terEncode` to prevent injection vulnerabilities and add related documentation. (apache#61662) (apache#61777) pick from apache#61662
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
- Added functionality to handle point queries using inverted indexes, allowing for efficient row lookups based on equality conditions. - Introduced new methods in the PointQueryExecutor to manage inverted index queries, including handling multiple tablets and integrating BloomFilter-based tablet pruning. - Updated the segment structure to include term zone maps for optimized query performance. - Enhanced the session variables and statement context to support inverted index query configurations. This change improves query performance and expands the capabilities of the query executor to utilize inverted indexes effectively.
AngryCucumber
requested review from
BiteTheDDDDt,
CalvinKirs,
dataroaring,
eldenmoon,
gavinchou,
morningman,
w41ter and
zclllyybb
as code owners
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Possible file(s) that should be tracked in LFS detected: 🚨The following file(s) exceeds the file size limit:
Consider using |
github-actions
bot
added
the
lfs-detected!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)