[enhance](load) only set brokerLoadBatchSize when enableMemtableOnSinkNode is true by sollhui · Pull Request #60301 · apache/doris

sollhui · 2026-01-28T07:54:51Z

What problem does this PR solve?

#36477 introduce brokerLoadBatchSize to improve performance for greater batch size. But for load path in VNodeChannel(enableMemtableOnSinkNode is false), the batching operation has been performed when sending the block, at the same time, memory issue was introduced: large memory occupation happened when serialize/deserialize block in wide-table scenarios.

Serialize block:

Deserialize block:

To solve this problem, only set brokerLoadBatchSize when enableMemtableOnSinkNode is true

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2026-01-28T07:54:57Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

sollhui · 2026-01-28T07:58:01Z

run buildall

sollhui · 2026-01-28T08:25:45Z

run buildall

sollhui · 2026-01-28T09:11:04Z

run buildall

doris-robot · 2026-01-28T09:43:16Z

TPC-H: Total hot run time: 32253 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5864b57ad333056c2de05add49ee11235cfbe7c3, data reload: false

------ Round 1 ----------------------------------
q1	17646	5184	5043	5043
q2	2129	299	229	229
q3	10121	1325	755	755
q4	10213	856	316	316
q5	7527	2168	1892	1892
q6	202	183	153	153
q7	905	730	585	585
q8	9259	1411	1069	1069
q9	5230	4864	4857	4857
q10	6771	1947	1575	1575
q11	512	282	264	264
q12	334	378	224	224
q13	17768	4048	3275	3275
q14	231	243	218	218
q15	867	828	819	819
q16	696	721	626	626
q17	633	806	468	468
q18	6795	6632	7593	6632
q19	1330	1064	655	655
q20	442	362	232	232
q21	2911	2312	2085	2085
q22	374	343	281	281
Total cold run time: 102896 ms
Total hot run time: 32253 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5628	5559	5600	5559
q2	292	354	264	264
q3	2458	2857	2628	2628
q4	1423	1820	1444	1444
q5	4780	4466	4555	4466
q6	240	197	149	149
q7	2139	1941	1778	1778
q8	2535	2371	2465	2371
q9	7650	7325	7611	7325
q10	2855	2989	2634	2634
q11	548	478	437	437
q12	676	716	591	591
q13	3858	4460	3381	3381
q14	274	288	272	272
q15	823	793	788	788
q16	642	701	640	640
q17	1081	1300	1305	1300
q18	7408	7443	7277	7277
q19	855	823	803	803
q20	1958	2033	1873	1873
q21	4470	4228	4026	4026
q22	563	563	491	491
Total cold run time: 53156 ms
Total hot run time: 50497 ms

doris-robot · 2026-01-28T10:00:04Z

ClickBench: Total hot run time: 28.31 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5864b57ad333056c2de05add49ee11235cfbe7c3, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.09	0.09
query4	1.61	0.12	0.11
query5	0.27	0.25	0.25
query6	1.17	0.67	0.67
query7	0.03	0.03	0.03
query8	0.06	0.04	0.03
query9	0.58	0.51	0.48
query10	0.55	0.55	0.55
query11	0.15	0.09	0.09
query12	0.14	0.10	0.11
query13	0.63	0.61	0.63
query14	1.06	1.06	1.06
query15	0.88	0.86	0.87
query16	0.37	0.38	0.39
query17	1.11	1.12	1.18
query18	0.23	0.22	0.21
query19	2.08	2.02	2.03
query20	0.02	0.01	0.01
query21	15.41	0.23	0.14
query22	5.13	0.06	0.05
query23	15.89	0.30	0.10
query24	1.68	0.74	0.26
query25	0.08	0.12	0.17
query26	0.14	0.14	0.13
query27	0.09	0.07	0.04
query28	4.66	1.13	0.96
query29	12.55	3.89	3.17
query30	0.28	0.13	0.12
query31	2.83	0.64	0.40
query32	3.22	0.59	0.50
query33	3.16	3.22	3.26
query34	16.32	5.41	4.67
query35	4.76	4.79	4.82
query36	0.66	0.51	0.49
query37	0.11	0.07	0.07
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.19	0.16	0.14
query41	0.10	0.04	0.03
query42	0.05	0.03	0.03
query43	0.05	0.04	0.04
Total cold run time: 98.83 s
Total hot run time: 28.31 s

hello-stephen · 2026-01-28T10:37:35Z

FE UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-01-28T12:01:01Z

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

liaoxin01

LGTM

github-actions · 2026-01-28T12:48:52Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-01-28T12:48:54Z

PR approved by anyone and no changes requested.

sollhui · 2026-01-28T14:10:17Z

run buildall

sollhui · 2026-01-28T14:12:15Z

run buildall

doris-robot · 2026-01-28T14:34:25Z

TPC-H: Total hot run time: 32180 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 01bf8cb83e165451a4d9f31e075b306118c2ba46, data reload: false

------ Round 1 ----------------------------------
q1	17620	5246	5061	5061
q2	2084	313	192	192
q3	10184	1330	764	764
q4	10228	904	328	328
q5	8445	2210	1922	1922
q6	220	181	148	148
q7	888	752	605	605
q8	9279	1396	1213	1213
q9	5338	4869	4959	4869
q10	6894	1961	1584	1584
q11	517	288	270	270
q12	387	383	217	217
q13	17822	4100	3217	3217
q14	253	245	227	227
q15	910	830	811	811
q16	682	686	622	622
q17	645	783	515	515
q18	6814	6553	6481	6481
q19	1230	992	628	628
q20	403	346	231	231
q21	2684	2021	2004	2004
q22	365	318	271	271
Total cold run time: 103892 ms
Total hot run time: 32180 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5328	5291	5307	5291
q2	257	345	247	247
q3	2160	2692	2257	2257
q4	1361	1767	1350	1350
q5	4270	4288	4373	4288
q6	213	182	140	140
q7	1944	1847	1674	1674
q8	2463	2323	2309	2309
q9	7039	6949	6984	6949
q10	2606	2835	2424	2424
q11	532	459	448	448
q12	628	724	562	562
q13	3546	4071	3222	3222
q14	295	292	270	270
q15	846	801	816	801
q16	658	701	652	652
q17	1100	1264	1334	1264
q18	7423	7312	7185	7185
q19	841	820	800	800
q20	1977	2051	1899	1899
q21	4605	4350	4122	4122
q22	590	543	509	509
Total cold run time: 50682 ms
Total hot run time: 48663 ms

doris-robot · 2026-01-28T14:51:01Z

ClickBench: Total hot run time: 28.22 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 01bf8cb83e165451a4d9f31e075b306118c2ba46, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.27	0.09	0.09
query4	1.61	0.12	0.11
query5	0.28	0.25	0.25
query6	1.17	0.67	0.67
query7	0.04	0.03	0.03
query8	0.06	0.04	0.04
query9	0.57	0.50	0.49
query10	0.55	0.55	0.55
query11	0.13	0.10	0.10
query12	0.13	0.10	0.10
query13	0.63	0.62	0.61
query14	1.07	1.06	1.05
query15	0.87	0.89	0.87
query16	0.39	0.39	0.39
query17	1.16	1.16	1.12
query18	0.23	0.21	0.21
query19	2.01	1.93	2.09
query20	0.02	0.01	0.01
query21	15.42	0.24	0.13
query22	5.16	0.05	0.05
query23	15.79	0.29	0.10
query24	1.45	0.27	0.48
query25	0.08	0.06	0.06
query26	0.14	0.14	0.13
query27	0.08	0.06	0.06
query28	3.92	1.16	0.97
query29	12.56	3.92	3.15
query30	0.28	0.14	0.11
query31	2.84	0.64	0.41
query32	3.24	0.60	0.49
query33	3.23	3.24	3.33
query34	16.35	5.49	4.74
query35	4.78	4.72	4.80
query36	0.66	0.51	0.49
query37	0.11	0.07	0.07
query38	0.08	0.04	0.04
query39	0.04	0.03	0.04
query40	0.17	0.18	0.15
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 97.9 s
Total hot run time: 28.22 s

sollhui · 2026-01-28T14:57:02Z

run buildall

doris-robot · 2026-01-28T15:19:41Z

TPC-H: Total hot run time: 31974 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 64eca8433414db0ff02d5720e9e5719b1ea6bb55, data reload: false

------ Round 1 ----------------------------------
q1	17614	5262	5032	5032
q2	2037	341	189	189
q3	10185	1336	773	773
q4	10216	885	309	309
q5	7533	2195	1896	1896
q6	196	182	152	152
q7	877	722	597	597
q8	9252	1403	1147	1147
q9	5136	4856	4723	4723
q10	6794	1941	1574	1574
q11	518	276	279	276
q12	352	385	224	224
q13	17780	4095	3228	3228
q14	244	253	221	221
q15	880	827	805	805
q16	666	697	621	621
q17	633	803	474	474
q18	6900	6454	7489	6454
q19	1268	1064	637	637
q20	398	371	237	237
q21	2907	2334	2113	2113
q22	390	334	292	292
Total cold run time: 102776 ms
Total hot run time: 31974 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5690	5539	5530	5530
q2	270	351	270	270
q3	2365	2816	2419	2419
q4	1459	2000	1384	1384
q5	4556	4688	4557	4557
q6	233	190	151	151
q7	2080	1923	1793	1793
q8	2605	2463	2510	2463
q9	7436	7364	7561	7364
q10	2837	3088	2632	2632
q11	558	544	459	459
q12	661	729	616	616
q13	3830	4362	3266	3266
q14	264	287	262	262
q15	830	796	780	780
q16	641	677	633	633
q17	1081	1263	1277	1263
q18	7601	7283	7487	7283
q19	819	826	826	826
q20	2004	2057	1897	1897
q21	4511	4197	4039	4039
q22	558	551	518	518
Total cold run time: 52889 ms
Total hot run time: 50405 ms

doris-robot · 2026-01-28T15:36:26Z

ClickBench: Total hot run time: 28.41 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 64eca8433414db0ff02d5720e9e5719b1ea6bb55, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.04	0.04
query3	0.26	0.08	0.08
query4	1.60	0.11	0.11
query5	0.28	0.26	0.25
query6	1.18	0.68	0.69
query7	0.03	0.03	0.03
query8	0.05	0.04	0.04
query9	0.57	0.50	0.50
query10	0.56	0.56	0.56
query11	0.15	0.10	0.10
query12	0.14	0.10	0.11
query13	0.62	0.62	0.60
query14	1.06	1.08	1.06
query15	0.87	0.86	0.87
query16	0.39	0.40	0.40
query17	1.11	1.08	1.13
query18	0.23	0.21	0.22
query19	2.01	2.04	2.08
query20	0.03	0.02	0.02
query21	15.40	0.27	0.14
query22	5.09	0.06	0.04
query23	15.90	0.27	0.10
query24	1.54	0.36	0.83
query25	0.10	0.12	0.07
query26	0.14	0.13	0.12
query27	0.06	0.05	0.06
query28	4.67	1.15	0.97
query29	12.56	3.97	3.18
query30	0.28	0.12	0.11
query31	2.81	0.65	0.39
query32	3.24	0.59	0.51
query33	3.28	3.31	3.27
query34	16.19	5.36	4.67
query35	4.80	4.82	4.75
query36	0.64	0.50	0.49
query37	0.12	0.08	0.07
query38	0.08	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.16	0.15
query41	0.10	0.03	0.03
query42	0.05	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 98.61 s
Total hot run time: 28.41 s

hello-stephen · 2026-01-28T16:09:04Z

FE UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉
Increment coverage report
Complete coverage report

doris-robot · 2026-01-28T16:13:14Z

BE UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.75% (19268/36526)
Line Coverage	36.11% (178966/495665)
Region Coverage	32.56% (138829/426362)
Branch Coverage	33.49% (60066/179341)

hello-stephen · 2026-01-28T18:55:43Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.49% (25596/35804)
Line Coverage	54.07% (267353/494432)
Region Coverage	51.64% (222405/430680)
Branch Coverage	53.11% (95624/180057)

hello-stephen · 2026-01-28T19:06:57Z

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

sollhui · 2026-01-29T07:42:55Z

run buildall

sollhui · 2026-01-29T08:58:31Z

run buildall

hello-stephen · 2026-01-29T12:09:32Z

BE UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.75% (19277/36541)
Line Coverage	36.14% (179159/495761)
Region Coverage	32.60% (138978/426375)
Branch Coverage	33.52% (60114/179337)

hello-stephen · 2026-01-29T13:25:21Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	69.99% (25068/35819)
Line Coverage	52.43% (259296/494528)
Region Coverage	49.73% (214180/430693)
Branch Coverage	51.08% (91964/180053)

hello-stephen · 2026-01-29T15:11:45Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	69.99% (25068/35819)
Line Coverage	52.43% (259296/494528)
Region Coverage	49.73% (214180/430693)
Branch Coverage	51.08% (91964/180053)

sollhui · 2026-01-29T15:18:36Z

run buildall

doris-robot · 2026-01-29T15:43:08Z

TPC-H: Total hot run time: 31888 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9834a44f4341fa72c9b855dc5db83c3cd6bccdd9, data reload: false

------ Round 1 ----------------------------------
q1	16665	5210	5125	5125
q2	2065	345	198	198
q3	10239	1310	756	756
q4	10397	806	317	317
q5	7578	2202	1901	1901
q6	202	184	155	155
q7	886	741	601	601
q8	9266	1396	1112	1112
q9	5224	4867	4817	4817
q10	6788	1953	1555	1555
q11	513	313	306	306
q12	350	383	234	234
q13	17770	4120	3222	3222
q14	239	254	224	224
q15	897	844	814	814
q16	699	693	614	614
q17	645	811	452	452
q18	7033	6718	6518	6518
q19	1858	999	610	610
q20	400	347	237	237
q21	2706	2044	1833	1833
q22	361	331	287	287
Total cold run time: 102781 ms
Total hot run time: 31888 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5469	5306	5343	5306
q2	281	331	266	266
q3	2167	2698	2272	2272
q4	1361	1752	1301	1301
q5	4333	4212	4256	4212
q6	224	189	144	144
q7	1906	1868	1702	1702
q8	2508	2323	2271	2271
q9	6995	7074	7027	7027
q10	2614	2801	2426	2426
q11	525	467	447	447
q12	611	710	560	560
q13	3557	4019	3261	3261
q14	276	307	270	270
q15	841	803	817	803
q16	652	683	644	644
q17	1094	1226	1282	1226
q18	7406	7459	7158	7158
q19	832	825	796	796
q20	1973	2055	1889	1889
q21	4647	4278	4178	4178
q22	577	562	528	528
Total cold run time: 50849 ms
Total hot run time: 48687 ms

doris-robot · 2026-01-29T15:59:57Z

ClickBench: Total hot run time: 28.1 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9834a44f4341fa72c9b855dc5db83c3cd6bccdd9, data reload: false

query1	0.05	0.05	0.05
query2	0.09	0.05	0.05
query3	0.27	0.09	0.08
query4	1.60	0.11	0.11
query5	0.26	0.25	0.24
query6	1.16	0.67	0.67
query7	0.03	0.02	0.03
query8	0.05	0.04	0.04
query9	0.57	0.50	0.52
query10	0.55	0.53	0.54
query11	0.13	0.09	0.10
query12	0.14	0.10	0.10
query13	0.62	0.62	0.61
query14	1.06	1.05	1.06
query15	0.88	0.85	0.86
query16	0.41	0.40	0.41
query17	1.14	1.13	1.08
query18	0.23	0.21	0.21
query19	1.99	1.98	2.04
query20	0.02	0.01	0.01
query21	15.41	0.24	0.14
query22	5.06	0.05	0.05
query23	15.78	0.29	0.10
query24	1.39	0.60	0.18
query25	0.12	0.07	0.07
query26	0.15	0.13	0.14
query27	0.08	0.05	0.05
query28	4.24	1.14	0.96
query29	12.53	3.90	3.18
query30	0.28	0.15	0.12
query31	2.82	0.63	0.40
query32	3.27	0.58	0.49
query33	3.25	3.23	3.30
query34	16.37	5.40	4.71
query35	4.82	4.82	4.76
query36	0.65	0.49	0.49
query37	0.11	0.07	0.07
query38	0.08	0.05	0.03
query39	0.05	0.03	0.03
query40	0.18	0.16	0.15
query41	0.09	0.03	0.02
query42	0.05	0.03	0.03
query43	0.05	0.04	0.04
Total cold run time: 98.08 s
Total hot run time: 28.1 s

hello-stephen · 2026-01-29T16:47:02Z

BE UT Coverage Report

Increment line coverage 28.57% (2/7) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.71% (19263/36543)
Line Coverage	36.11% (179012/495780)
Region Coverage	32.58% (138924/426437)
Branch Coverage	33.49% (60066/179345)

hello-stephen · 2026-01-29T19:06:22Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (7/7) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.44% (25590/35821)
Line Coverage	54.05% (267325/494547)
Region Coverage	51.66% (222518/430755)
Branch Coverage	53.08% (95584/180061)

hello-stephen · 2026-01-29T19:20:31Z

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

sollhui · 2026-01-30T09:33:51Z

run buildall

doris-robot · 2026-01-30T10:14:10Z

TPC-H: Total hot run time: 31955 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e7b1943cb0a139f4e80b5bef6065b75805c1d85b, data reload: false

------ Round 1 ----------------------------------
q1	17640	5314	5072	5072
q2	2067	306	197	197
q3	10212	1274	761	761
q4	10231	900	318	318
q5	7612	2128	1923	1923
q6	211	183	149	149
q7	889	743	602	602
q8	9262	1448	1078	1078
q9	5220	4744	4807	4744
q10	6820	1941	1564	1564
q11	487	288	267	267
q12	335	371	228	228
q13	17798	4034	3215	3215
q14	248	233	225	225
q15	897	817	819	817
q16	676	675	629	629
q17	686	835	437	437
q18	6575	6570	7415	6570
q19	2062	1172	665	665
q20	403	372	235	235
q21	2824	2270	1964	1964
q22	383	339	295	295
Total cold run time: 103538 ms
Total hot run time: 31955 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5597	5469	5484	5469
q2	261	340	243	243
q3	2330	2880	2431	2431
q4	1484	1924	1440	1440
q5	5022	4622	4503	4503
q6	225	188	140	140
q7	2096	2010	1825	1825
q8	2553	2399	2494	2399
q9	7465	7430	7589	7430
q10	2831	3132	2903	2903
q11	535	475	440	440
q12	701	785	657	657
q13	3666	4034	3209	3209
q14	271	282	262	262
q15	831	801	793	793
q16	635	681	639	639
q17	1049	1273	1371	1273
q18	7555	7526	7269	7269
q19	802	762	765	762
q20	1976	2129	1888	1888
q21	4486	4181	4114	4114
q22	569	540	494	494
Total cold run time: 52940 ms
Total hot run time: 50583 ms

doris-robot · 2026-01-30T10:30:55Z

ClickBench: Total hot run time: 28.23 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e7b1943cb0a139f4e80b5bef6065b75805c1d85b, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.04	0.04
query3	0.25	0.08	0.09
query4	1.60	0.12	0.11
query5	0.27	0.26	0.24
query6	1.16	0.70	0.67
query7	0.03	0.02	0.02
query8	0.06	0.03	0.04
query9	0.56	0.48	0.49
query10	0.56	0.55	0.54
query11	0.14	0.09	0.09
query12	0.14	0.10	0.10
query13	0.63	0.62	0.60
query14	1.07	1.05	1.05
query15	0.87	0.87	0.87
query16	0.40	0.38	0.38
query17	1.08	1.12	1.09
query18	0.23	0.21	0.20
query19	1.95	2.02	1.97
query20	0.02	0.02	0.01
query21	15.44	0.26	0.14
query22	5.23	0.05	0.05
query23	15.95	0.27	0.11
query24	0.94	0.56	0.22
query25	0.08	0.07	0.06
query26	0.14	0.14	0.13
query27	0.08	0.07	0.06
query28	3.88	1.14	0.96
query29	12.58	3.95	3.19
query30	0.27	0.14	0.11
query31	2.82	0.63	0.40
query32	3.24	0.59	0.49
query33	3.20	3.31	3.29
query34	15.91	5.41	4.75
query35	4.75	4.79	4.83
query36	0.66	0.49	0.50
query37	0.11	0.06	0.06
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.15	0.15
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 96.91 s
Total hot run time: 28.23 s

hello-stephen · 2026-01-30T12:36:00Z

FE Regression Coverage Report

Increment line coverage 16.67% (2/12) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-02-02T04:10:51Z

FE Regression Coverage Report

Increment line coverage 16.67% (2/12) 🎉
Increment coverage report
Complete coverage report

liaoxin01

LGTM

github-actions · 2026-02-04T03:32:04Z

PR approved by at least one committer and no changes requested.

…kNode is true (#60301) ### What problem does this PR solve? #36477 introduce brokerLoadBatchSize to improve performance for greater batch size. But for load path in `VNodeChannel`(`enableMemtableOnSinkNode` is false), the batching operation has been performed when sending the block, at the same time, memory issue was introduced: large memory occupation happened when serialize/deserialize block in wide-table scenarios. **Serialize block:** <img width="312" height="588" alt="image" src="https://github.com/user-attachments/assets/e63390cb-83ac-4cd8-b171-6fb6fc5a2785" /> **Deserialize block:** <img width="312" height="488" alt="image" src="https://github.com/user-attachments/assets/425d136d-21b0-4c8d-a29f-1686d3de4fcb" /> To solve this problem, only set brokerLoadBatchSize when enableMemtableOnSinkNode is true

…mtableOnSinkNode is true (#62089) pick #60301 --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

sollhui force-pushed the opt_batch_size branch from 13593a8 to d627f30 Compare January 28, 2026 08:25

sollhui force-pushed the opt_batch_size branch from d627f30 to 5864b57 Compare January 28, 2026 08:43

liaoxin01 previously approved these changes Jan 28, 2026

View reviewed changes

liaoxin01 added the dev/4.0.x label Jan 28, 2026

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 28, 2026

github-actions bot added the reviewed label Jan 28, 2026

sollhui dismissed liaoxin01’s stale review via baac91b January 28, 2026 14:10

sollhui force-pushed the opt_batch_size branch from 5864b57 to baac91b Compare January 28, 2026 14:10

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jan 28, 2026

sollhui force-pushed the opt_batch_size branch from 01bf8cb to 64eca84 Compare January 28, 2026 14:56

sollhui force-pushed the opt_batch_size branch from 3d96876 to abe0ec6 Compare January 29, 2026 08:58

only set brokerLoadBatchSize when enableMemtableOnSinkNode is true

e7b1943

sollhui force-pushed the opt_batch_size branch from 1aa5fca to e7b1943 Compare January 30, 2026 09:25

liaoxin01 approved these changes Feb 4, 2026

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 4, 2026

JNSimba approved these changes Feb 5, 2026

View reviewed changes

JNSimba merged commit e1a7abe into apache:master Feb 5, 2026
29 of 30 checks passed

github-actions bot mentioned this pull request Feb 5, 2026

branch-4.0: [enhance](load) only set brokerLoadBatchSize when enableMemtableOnSinkNode is true #60301 #60524

Open

sollhui mentioned this pull request Apr 3, 2026

branch-4.1: [enhance](load)only set brokerLoadBatchSize when enableMemtableOnSinkNode is true #62089

Merged

yiguolei pushed a commit that referenced this pull request Apr 3, 2026

branch-4.1: [enhance](load)only set brokerLoadBatchSize when enableMe…

8003eea

…mtableOnSinkNode is true (#62089) pick #60301 --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

yiguolei added the dev/4.1.0-merged label Apr 3, 2026

Conversation

sollhui commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Jan 28, 2026

Uh oh!

sollhui commented Jan 28, 2026

Uh oh!

sollhui commented Jan 28, 2026

Uh oh!

sollhui commented Jan 28, 2026

Uh oh!

doris-robot commented Jan 28, 2026

Uh oh!

doris-robot commented Jan 28, 2026

Uh oh!

hello-stephen commented Jan 28, 2026

FE UT Coverage Report

Uh oh!

hello-stephen commented Jan 28, 2026

FE Regression Coverage Report

Uh oh!

liaoxin01 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 28, 2026

Uh oh!

github-actions bot commented Jan 28, 2026

Uh oh!

sollhui commented Jan 28, 2026

Uh oh!

sollhui commented Jan 28, 2026

Uh oh!

doris-robot commented Jan 28, 2026

Uh oh!

doris-robot commented Jan 28, 2026

Uh oh!

sollhui commented Jan 28, 2026

Uh oh!

doris-robot commented Jan 28, 2026

Uh oh!

doris-robot commented Jan 28, 2026

Uh oh!

hello-stephen commented Jan 28, 2026

FE UT Coverage Report

Uh oh!

doris-robot commented Jan 28, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Jan 28, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Jan 28, 2026

FE Regression Coverage Report

Uh oh!

sollhui commented Jan 29, 2026

Uh oh!

sollhui commented Jan 29, 2026

Uh oh!

hello-stephen commented Jan 29, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Jan 29, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Jan 29, 2026

BE Regression && UT Coverage Report

Uh oh!

sollhui commented Jan 29, 2026

Uh oh!

doris-robot commented Jan 29, 2026

Uh oh!

doris-robot commented Jan 29, 2026

Uh oh!

hello-stephen commented Jan 29, 2026

BE UT Coverage Report

sollhui commented Jan 28, 2026 •

edited

Loading