Skip to content

[feature](partition prune) Add variable skip_prune_predicate to skip prune predicate after partition prune #57688

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Nov 5, 2025
Merged

[feature](partition prune) Add variable skip_prune_predicate to skip prune predicate after partition prune #57688

merged 3 commits into from
Nov 5, 2025

Conversation

@feiniaofeiafei
Copy link
Contributor

@feiniaofeiafei feiniaofeiafei commented Nov 4, 2025 •
edited
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #57169
Add session variable skip_prune_predicate to skip prune predicate after partition prune.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Nov 4, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@feiniaofeiafei
Copy link
Contributor Author

feiniaofeiafei commented Nov 4, 2025

run buildall

@feiniaofeiafei
Copy link
Contributor Author

feiniaofeiafei commented Nov 4, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Nov 4, 2025

TPC-DS: Total hot run time: 190226 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b1863de99a04b5bd5e9a21c6556106bbaf2e0f18, data reload: false

query1	1045	417	400	400
query2	6560	1729	1683	1683
query3	6761	219	221	219
query4	26617	23768	23562	23562
query5	4909	633	484	484
query6	347	262	237	237
query7	4658	501	304	304
query8	317	265	258	258
query9	8755	2605	2606	2605
query10	523	359	294	294
query11	16083	15155	14892	14892
query12	203	121	121	121
query13	1704	577	446	446
query14	12151	9351	9380	9351
query15	206	191	188	188
query16	7676	688	533	533
query17	1603	805	701	701
query18	2088	459	384	384
query19	240	242	206	206
query20	139	128	127	127
query21	222	138	118	118
query22	4708	4651	4338	4338
query23	34501	34149	33773	33773
query24	8271	2504	2544	2504
query25	592	524	490	490
query26	1255	278	168	168
query27	3196	546	370	370
query28	4800	2266	2282	2266
query29	815	630	521	521
query30	330	233	202	202
query31	924	842	747	747
query32	84	73	71	71
query33	590	448	335	335
query34	822	861	537	537
query35	823	875	800	800
query36	1017	1039	920	920
query37	124	121	92	92
query38	3684	3602	3692	3602
query39	1520	1456	1413	1413
query40	216	128	120	120
query41	60	56	56	56
query42	119	108	112	108
query43	494	503	488	488
query44	1231	755	753	753
query45	185	178	170	170
query46	891	1003	639	639
query47	1756	1783	1700	1700
query48	410	430	310	310
query49	742	493	420	420
query50	642	682	414	414
query51	3896	3995	3958	3958
query52	111	112	106	106
query53	234	262	205	205
query54	321	286	270	270
query55	87	84	81	81
query56	319	301	306	301
query57	1176	1207	1123	1123
query58	287	277	273	273
query59	2617	2678	2562	2562
query60	343	341	326	326
query61	160	158	155	155
query62	803	738	671	671
query63	239	194	192	192
query64	4406	1172	859	859
query65	4070	3976	3938	3938
query66	1094	425	326	326
query67	15401	15182	14897	14897
query68	8552	925	590	590
query69	493	336	297	297
query70	1293	1295	1297	1295
query71	492	335	313	313
query72	5840	5006	5014	5006
query73	701	596	362	362
query74	8911	9029	8924	8924
query75	4123	3292	2762	2762
query76	3820	1151	725	725
query77	827	395	315	315
query78	9420	9720	8989	8989
query79	2004	836	582	582
query80	629	553	497	497
query81	486	264	229	229
query82	435	157	133	133
query83	262	260	241	241
query84	249	117	102	102
query85	941	490	433	433
query86	336	319	298	298
query87	3703	3800	3595	3595
query88	3191	2231	2220	2220
query89	388	320	292	292
query90	1988	217	207	207
query91	156	161	141	141
query92	80	68	66	66
query93	1143	959	638	638
query94	699	443	324	324
query95	403	311	297	297
query96	484	590	275	275
query97	2933	2965	2884	2884
query98	234	217	220	217
query99	1427	1433	1294	1294
Total cold run time: 279816 ms
Total hot run time: 190226 ms

@doris-robot
Copy link

doris-robot commented Nov 4, 2025

ClickBench: Total hot run time: 27.65 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b1863de99a04b5bd5e9a21c6556106bbaf2e0f18, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.05	0.04
query3	0.26	0.08	0.09
query4	1.60	0.12	0.12
query5	0.28	0.26	0.26
query6	1.17	0.65	0.63
query7	0.03	0.03	0.03
query8	0.05	0.04	0.04
query9	0.60	0.54	0.51
query10	0.58	0.57	0.56
query11	0.16	0.14	0.12
query12	0.16	0.12	0.12
query13	0.62	0.60	0.60
query14	1.00	1.01	1.00
query15	0.85	0.82	0.84
query16	0.39	0.39	0.39
query17	1.03	1.03	0.99
query18	0.22	0.23	0.20
query19	1.93	1.81	1.75
query20	0.01	0.01	0.01
query21	15.45	0.19	0.13
query22	5.08	0.07	0.05
query23	15.66	0.26	0.10
query24	3.15	0.83	0.45
query25	0.07	0.06	0.07
query26	0.14	0.12	0.12
query27	0.07	0.05	0.05
query28	4.80	1.13	0.94
query29	12.56	3.97	3.29
query30	0.27	0.13	0.12
query31	2.84	0.59	0.38
query32	3.23	0.55	0.47
query33	3.03	3.11	3.14
query34	15.88	5.17	4.64
query35	4.61	4.57	4.60
query36	0.69	0.50	0.49
query37	0.10	0.07	0.07
query38	0.07	0.04	0.04
query39	0.03	0.04	0.03
query40	0.17	0.15	0.14
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 99.15 s
Total hot run time: 27.65 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 4, 2025

FE UT Coverage Report

Increment line coverage 50.00% (2/4) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 4, 2025

FE Regression Coverage Report

Increment line coverage 100.00% (4/4) 🎉
Increment coverage report
Complete coverage report

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 5, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Nov 5, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 5, 2025

PR approved by anyone and no changes requested.

@924060929 924060929 merged commit 7631f2b into apache:master Nov 5, 2025
30 checks passed
feiniaofeiafei added a commit to feiniaofeiafei/doris that referenced this pull request Nov 14, 2025
@feiniaofeiafei
[feature](partition prune) Add variable skip_prune_predicate to skip …
9d1bcf5 
…prune predicate after partition prune (apache#57688)

Add session variable skip_prune_predicate to skip prune predicate after
partition prune.

Related PR: apache#57169
@feiniaofeiafei feiniaofeiafei mentioned this pull request Nov 14, 2025
Merged
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Nov 18, 2025
@feiniaofeiafei @wyxxxcat
[feature](partition prune) Add variable skip_prune_predicate to skip …
0dd5395 
…prune predicate after partition prune (apache#57688)

Add session variable skip_prune_predicate to skip prune predicate after
partition prune.

Related PR: apache#57169
feiniaofeiafei added a commit to feiniaofeiafei/doris that referenced this pull request Nov 25, 2025
@feiniaofeiafei
[feature](partition prune) Add variable skip_prune_predicate to skip …
63f4681 
…prune predicate after partition prune (apache#57688)

Add session variable skip_prune_predicate to skip prune predicate after
partition prune.

Related PR: apache#57169
yiguolei pushed a commit that referenced this pull request Nov 27, 2025
@feiniaofeiafei
branch-4.0: [feature](partition prune) support remove predicates that…
2a26480 
… are always true after partition prune in list partition #57169 #57688 #57817 (#58040)

picked from #57169 #57688 #57817
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@924060929 924060929 924060929 approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.2-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@feiniaofeiafei @Thearas @doris-robot @hello-stephen @924060929 @morrySnow @yiguolei