Skip to content

Conversation

@linrrzqqq
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

The construction cost of civil_second in cctz is expensive, details can be seen: #57423.

In some cases, we do not need to construct the complete civil_time, but only the UTC offset is required, like:HOUR(FROM_UNIXTIME(ts))

This PR supports the lookup_offset function in cctz, which has the same return type as lookup, but the civil_second is empty, avoiding the expensive construction cost.

return {{}, utc_offset, is_dst, abbreviations[abbr_index]};

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Nov 13, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@linrrzqqq
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34794 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f8ce9c59408c770adfb9815da07c60cf6695ef5b, data reload: false

------ Round 1 ----------------------------------
q1	17614	5206	5038	5038
q2	2038	304	192	192
q3	10266	1310	710	710
q4	10243	955	377	377
q5	7549	2472	2332	2332
q6	189	167	135	135
q7	931	761	629	629
q8	9354	1416	1064	1064
q9	7030	5277	5233	5233
q10	6918	2250	1852	1852
q11	496	307	296	296
q12	364	382	240	240
q13	18093	3632	3022	3022
q14	229	277	212	212
q15	586	510	509	509
q16	1027	1010	932	932
q17	580	873	368	368
q18	7691	7456	7826	7456
q19	1517	985	562	562
q20	356	371	228	228
q21	4173	3493	2371	2371
q22	1116	1086	1036	1036
Total cold run time: 108360 ms
Total hot run time: 34794 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5413	5392	5357	5357
q2	239	338	241	241
q3	2476	2881	2479	2479
q4	1391	1834	1529	1529
q5	4535	4357	4463	4357
q6	220	174	130	130
q7	1956	1941	1803	1803
q8	2603	2495	2504	2495
q9	7471	7196	7315	7196
q10	2987	3087	2655	2655
q11	560	490	492	490
q12	675	728	583	583
q13	3327	3632	3043	3043
q14	271	286	264	264
q15	527	511	500	500
q16	1026	1042	1015	1015
q17	1093	1454	1352	1352
q18	7261	7079	7209	7079
q19	765	772	919	772
q20	1935	1929	1822	1822
q21	4739	4331	4334	4331
q22	1115	1055	1018	1018
Total cold run time: 52585 ms
Total hot run time: 50511 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187525 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f8ce9c59408c770adfb9815da07c60cf6695ef5b, data reload: false

query1	1056	391	390	390
query2	6544	1720	1699	1699
query3	6758	220	221	220
query4	26515	23734	23007	23007
query5	4520	623	497	497
query6	353	251	237	237
query7	4645	498	304	304
query8	332	261	253	253
query9	8757	2561	2557	2557
query10	474	348	287	287
query11	15904	14995	14858	14858
query12	171	118	115	115
query13	1692	577	450	450
query14	11326	9287	9216	9216
query15	198	185	174	174
query16	7673	689	499	499
query17	1233	743	608	608
query18	2021	416	312	312
query19	213	193	173	173
query20	131	121	118	118
query21	220	133	113	113
query22	4007	4193	3996	3996
query23	33717	32916	33004	32916
query24	8465	2408	2392	2392
query25	590	523	443	443
query26	1234	273	158	158
query27	2761	496	346	346
query28	4392	2204	2193	2193
query29	838	600	488	488
query30	292	225	192	192
query31	915	802	728	728
query32	83	75	76	75
query33	590	374	322	322
query34	800	846	517	517
query35	805	821	765	765
query36	973	1006	910	910
query37	120	110	87	87
query38	3474	3528	3466	3466
query39	1462	1407	1395	1395
query40	221	126	116	116
query41	62	59	68	59
query42	129	116	114	114
query43	520	498	474	474
query44	1212	743	737	737
query45	186	180	170	170
query46	871	988	633	633
query47	1776	1766	1674	1674
query48	408	426	340	340
query49	777	504	425	425
query50	645	683	405	405
query51	3931	4026	4049	4026
query52	111	115	109	109
query53	249	272	194	194
query54	316	286	278	278
query55	83	85	92	85
query56	333	361	314	314
query57	1193	1192	1103	1103
query58	296	282	273	273
query59	2439	2716	2453	2453
query60	339	349	330	330
query61	170	158	161	158
query62	805	720	674	674
query63	221	196	189	189
query64	4443	1152	852	852
query65	4028	3957	3907	3907
query66	1088	442	336	336
query67	15325	15242	14992	14992
query68	4676	970	601	601
query69	484	327	283	283
query70	1378	1289	1208	1208
query71	447	343	329	329
query72	6074	5181	5330	5181
query73	658	600	369	369
query74	9145	8967	8890	8890
query75	3335	3313	2816	2816
query76	3333	1154	738	738
query77	567	439	320	320
query78	9607	9748	8923	8923
query79	2222	816	591	591
query80	1704	571	512	512
query81	572	261	228	228
query82	402	160	132	132
query83	360	261	242	242
query84	253	117	92	92
query85	931	485	428	428
query86	478	318	309	309
query87	3636	3671	3564	3564
query88	3116	2263	2244	2244
query89	396	331	285	285
query90	1977	228	214	214
query91	172	168	136	136
query92	79	66	69	66
query93	2644	1013	641	641
query94	777	446	335	335
query95	408	316	311	311
query96	489	582	282	282
query97	2915	2948	2859	2859
query98	256	224	217	217
query99	1311	1369	1283	1283
Total cold run time: 273233 ms
Total hot run time: 187525 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 27.92 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f8ce9c59408c770adfb9815da07c60cf6695ef5b, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.08	0.08
query4	1.62	0.11	0.11
query5	0.27	0.26	0.25
query6	1.18	0.64	0.65
query7	0.03	0.02	0.02
query8	0.06	0.04	0.04
query9	0.58	0.52	0.51
query10	0.59	0.58	0.57
query11	0.16	0.11	0.11
query12	0.15	0.12	0.11
query13	0.63	0.60	0.60
query14	1.00	1.00	0.99
query15	0.85	0.82	0.84
query16	0.40	0.38	0.41
query17	1.03	1.01	1.02
query18	0.21	0.20	0.20
query19	1.83	1.80	1.79
query20	0.01	0.01	0.01
query21	15.45	0.19	0.13
query22	5.11	0.07	0.05
query23	15.70	0.25	0.10
query24	2.98	0.96	1.25
query25	0.08	0.06	0.06
query26	0.15	0.13	0.13
query27	0.06	0.06	0.05
query28	5.82	1.14	0.94
query29	12.61	3.83	3.20
query30	0.29	0.14	0.12
query31	2.82	0.56	0.37
query32	3.23	0.56	0.47
query33	3.08	3.04	3.09
query34	15.86	5.17	4.55
query35	4.59	4.59	4.55
query36	0.67	0.52	0.49
query37	0.09	0.07	0.07
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.16	0.14
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 100.06 s
Total hot run time: 27.92 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.80% (18280/34618)
Line Coverage 38.17% (166198/435389)
Region Coverage 33.15% (129137/389557)
Branch Coverage 33.90% (55446/163556)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.57% (24346/34018)
Line Coverage 58.05% (253138/436065)
Region Coverage 53.51% (211418/395068)
Branch Coverage 54.73% (90103/164631)

@linrrzqqq
Copy link
Contributor Author

run p0

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.57% (24345/34018)
Line Coverage 58.05% (253146/436065)
Region Coverage 53.52% (211439/395068)
Branch Coverage 54.74% (90119/164631)

zclllyybb
zclllyybb previously approved these changes Nov 14, 2025
Copy link
Contributor

@zclllyybb zclllyybb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 14, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@linrrzqqq
Copy link
Contributor Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Nov 14, 2025
@doris-robot
Copy link

TPC-H: Total hot run time: 34174 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ccf363324c30327939b09411eead56ffb3b690f9, data reload: false

------ Round 1 ----------------------------------
q1	17605	5217	5073	5073
q2	2079	302	208	208
q3	10270	1276	701	701
q4	10241	960	371	371
q5	7569	2418	2317	2317
q6	181	168	133	133
q7	911	769	626	626
q8	9333	1301	1040	1040
q9	6923	5107	5119	5107
q10	6893	2236	1810	1810
q11	504	307	282	282
q12	335	362	236	236
q13	17818	3657	3062	3062
q14	237	243	226	226
q15	589	514	508	508
q16	1019	1005	948	948
q17	584	870	360	360
q18	7437	7173	7091	7091
q19	1098	941	536	536
q20	356	344	226	226
q21	3831	2550	2308	2308
q22	1061	1050	1005	1005
Total cold run time: 106874 ms
Total hot run time: 34174 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5054	5103	5145	5103
q2	255	337	235	235
q3	2169	2694	2287	2287
q4	1373	1753	1374	1374
q5	4179	4337	4469	4337
q6	210	164	134	134
q7	2116	2066	1824	1824
q8	2615	2556	2620	2556
q9	7350	7331	7378	7331
q10	3087	3261	2878	2878
q11	605	530	497	497
q12	677	772	653	653
q13	3752	3913	3314	3314
q14	287	324	295	295
q15	566	516	525	516
q16	1094	1128	1057	1057
q17	1175	1582	1375	1375
q18	7961	7817	7475	7475
q19	807	793	881	793
q20	2115	2098	1935	1935
q21	4973	4443	4248	4248
q22	1106	1065	1012	1012
Total cold run time: 53526 ms
Total hot run time: 51229 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 188174 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ccf363324c30327939b09411eead56ffb3b690f9, data reload: false

query1	1021	392	401	392
query2	6601	1723	1667	1667
query3	6760	225	221	221
query4	26809	23732	23412	23412
query5	4438	639	460	460
query6	335	238	220	220
query7	4639	491	297	297
query8	297	257	235	235
query9	8723	2625	2620	2620
query10	497	350	295	295
query11	15333	15152	14845	14845
query12	173	122	111	111
query13	1690	582	457	457
query14	10419	9413	9286	9286
query15	199	187	177	177
query16	7205	686	530	530
query17	1286	788	645	645
query18	2000	432	335	335
query19	214	207	191	191
query20	135	130	128	128
query21	218	135	114	114
query22	3977	4089	3922	3922
query23	34092	33063	33028	33028
query24	8515	2470	2430	2430
query25	609	549	479	479
query26	1250	274	162	162
query27	2757	503	347	347
query28	4387	2212	2196	2196
query29	882	645	513	513
query30	304	226	204	204
query31	907	842	716	716
query32	91	77	74	74
query33	602	396	337	337
query34	801	859	533	533
query35	818	847	809	809
query36	972	994	898	898
query37	114	102	84	84
query38	3527	3554	3442	3442
query39	1471	1443	1425	1425
query40	225	127	118	118
query41	63	60	59	59
query42	130	115	112	112
query43	482	494	454	454
query44	1214	749	748	748
query45	189	181	180	180
query46	879	981	636	636
query47	1731	1782	1713	1713
query48	387	429	321	321
query49	775	507	396	396
query50	647	678	409	409
query51	3829	3955	3848	3848
query52	117	108	106	106
query53	238	269	200	200
query54	293	288	273	273
query55	90	86	82	82
query56	330	304	301	301
query57	1215	1175	1113	1113
query58	286	274	271	271
query59	2545	2712	2643	2643
query60	343	407	333	333
query61	156	159	163	159
query62	824	724	686	686
query63	225	194	193	193
query64	4596	1174	865	865
query65	4029	3932	3953	3932
query66	1209	459	343	343
query67	15243	15111	14940	14940
query68	8528	928	597	597
query69	497	328	285	285
query70	1286	1267	1252	1252
query71	521	341	315	315
query72	5673	4943	4854	4854
query73	638	571	357	357
query74	8975	9124	8845	8845
query75	3998	3350	2806	2806
query76	3687	1136	722	722
query77	816	414	313	313
query78	9506	9827	8994	8994
query79	2041	813	600	600
query80	653	588	516	516
query81	492	257	229	229
query82	429	164	144	144
query83	273	263	252	252
query84	259	107	97	97
query85	921	477	440	440
query86	342	311	288	288
query87	3689	3721	3652	3652
query88	3697	2257	2255	2255
query89	393	332	293	293
query90	1937	228	225	225
query91	181	163	135	135
query92	89	66	64	64
query93	1486	977	653	653
query94	734	450	343	343
query95	411	312	314	312
query96	489	570	282	282
query97	2915	2946	2880	2880
query98	244	210	210	210
query99	1725	1448	1293	1293
Total cold run time: 274883 ms
Total hot run time: 188174 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 27.43 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ccf363324c30327939b09411eead56ffb3b690f9, data reload: false

query1	0.05	0.05	0.06
query2	0.09	0.05	0.05
query3	0.25	0.08	0.09
query4	1.61	0.11	0.12
query5	0.26	0.24	0.25
query6	1.16	0.64	0.64
query7	0.03	0.03	0.02
query8	0.06	0.04	0.05
query9	0.58	0.52	0.53
query10	0.60	0.58	0.57
query11	0.17	0.11	0.11
query12	0.15	0.12	0.12
query13	0.62	0.61	0.60
query14	1.00	1.01	1.00
query15	0.85	0.83	0.84
query16	0.39	0.42	0.39
query17	1.04	0.99	1.02
query18	0.21	0.20	0.20
query19	1.92	1.80	1.81
query20	0.02	0.02	0.01
query21	15.47	0.20	0.13
query22	5.11	0.07	0.05
query23	15.67	0.27	0.09
query24	3.10	0.53	0.75
query25	0.10	0.07	0.05
query26	0.14	0.13	0.13
query27	0.06	0.06	0.06
query28	4.66	1.16	0.93
query29	12.55	3.87	3.17
query30	0.29	0.14	0.10
query31	2.82	0.59	0.38
query32	3.24	0.54	0.47
query33	3.03	3.04	3.06
query34	15.63	5.15	4.56
query35	4.58	4.60	4.53
query36	0.67	0.49	0.48
query37	0.09	0.07	0.06
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.16	0.13	0.15
query41	0.09	0.04	0.04
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 98.72 s
Total hot run time: 27.43 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 14, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.81% (18285/34621)
Line Coverage 38.18% (166265/435440)
Region Coverage 33.19% (129297/389586)
Branch Coverage 33.91% (55468/163578)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.54% (24339/34021)
Line Coverage 58.04% (253130/436114)
Region Coverage 53.52% (211467/395097)
Branch Coverage 54.76% (90156/164653)

@linrrzqqq
Copy link
Contributor Author

run p0

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.51% (24328/34021)
Line Coverage 58.01% (252973/436114)
Region Coverage 53.51% (211417/395097)
Branch Coverage 54.72% (90104/164653)

Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman merged commit 4d2bbe9 into apache:master Nov 15, 2025
32 of 33 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 15, 2025
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

The construction cost of civil_second in cctz is expensive, details can
be seen: #57423.

In some cases, we do not need to construct the complete civil_time, but
only the UTC offset is required, like:`HOUR(FROM_UNIXTIME(ts))`

This PR supports the lookup_offset function in cctz, which has the same
return type as lookup, but the civil_second is empty, avoiding the
expensive construction cost.

```cpp
return {{}, utc_offset, is_dst, abbreviations[abbr_index]};
```
github-actions bot pushed a commit that referenced this pull request Nov 15, 2025
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

The construction cost of civil_second in cctz is expensive, details can
be seen: #57423.

In some cases, we do not need to construct the complete civil_time, but
only the UTC offset is required, like:`HOUR(FROM_UNIXTIME(ts))`

This PR supports the lookup_offset function in cctz, which has the same
return type as lookup, but the civil_second is empty, avoiding the
expensive construction cost.

```cpp
return {{}, utc_offset, is_dst, abbreviations[abbr_index]};
```
@linrrzqqq linrrzqqq deleted the cctz-offset-patch branch November 16, 2025 07:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.x dev/4.0.x reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants