Skip to content

[fix](meta)fix after the FE restarts, ConcurrentMap becomes LinkedHashMap #58382

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Nov 27, 2025
Merged

[fix](meta)fix after the FE restarts, ConcurrentMap becomes LinkedHashMap #58382

merged 1 commit into from
Nov 27, 2025

Conversation

@zddr
Copy link
Contributor

@zddr zddr commented Nov 26, 2025 •
edited
Loading

What problem does this PR solve?

For example, in the following case, after the FE restarts, snapshotInfos will become a LinkedHashMap:

@SerializedName("si")
private Map<Long, SnapshotInfo> snapshotInfos = Maps.newConcurrentMap();

The solution is to declare it as ConcurrentMap or, more specifically, ConcurrentHashMap:

@SerializedName("si")
private ConcurrentHashMap<Long, SnapshotInfo> snapshotInfos = Maps.newConcurrentMap();

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
fix after the FE restarts, ConcurrentMap becomes LinkedHashMap

Release note

fix after the FE restarts, ConcurrentMap becomes LinkedHashMap

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 26, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zddr
Copy link
Contributor Author

zddr commented Nov 26, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Nov 26, 2025

TPC-H: Total hot run time: 34594 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2abf36e577b257ff7bc8844418933390b3eca6da, data reload: false

------ Round 1 ----------------------------------
q1	17611	5054	4911	4911
q2	2037	335	213	213
q3	10215	1312	746	746
q4	10249	978	383	383
q5	7538	2408	2411	2408
q6	189	171	139	139
q7	936	801	644	644
q8	9379	1357	1083	1083
q9	7252	5369	5311	5311
q10	6908	2268	1823	1823
q11	505	296	281	281
q12	358	371	232	232
q13	17774	3667	3066	3066
q14	229	252	221	221
q15	612	517	509	509
q16	1068	1036	967	967
q17	604	879	376	376
q18	7386	7188	7184	7184
q19	1094	970	567	567
q20	350	347	228	228
q21	3814	3252	2336	2336
q22	1030	1017	966	966
Total cold run time: 107138 ms
Total hot run time: 34594 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4968	4920	4931	4920
q2	328	393	312	312
q3	2228	2708	2300	2300
q4	1370	1834	1357	1357
q5	4263	4630	4637	4630
q6	217	181	135	135
q7	2081	2013	1848	1848
q8	2672	2634	2700	2634
q9	7685	7794	7485	7485
q10	3124	3268	2866	2866
q11	595	537	507	507
q12	680	787	634	634
q13	3566	4008	3386	3386
q14	307	308	268	268
q15	544	553	530	530
q16	1105	1142	1068	1068
q17	1211	1548	1512	1512
q18	7866	7691	7613	7613
q19	852	855	898	855
q20	2023	2093	2020	2020
q21	5065	4421	4376	4376
q22	1094	1048	988	988
Total cold run time: 53844 ms
Total hot run time: 52244 ms

@doris-robot
Copy link

doris-robot commented Nov 26, 2025

TPC-DS: Total hot run time: 184320 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2abf36e577b257ff7bc8844418933390b3eca6da, data reload: false

query1	1076	398	402	398
query2	6556	1566	1590	1566
query3	6757	232	216	216
query4	26105	22853	22290	22290
query5	4411	628	477	477
query6	349	237	225	225
query7	4649	510	307	307
query8	295	260	260	260
query9	8702	2582	2554	2554
query10	535	374	330	330
query11	15467	14838	14561	14561
query12	175	121	113	113
query13	1674	576	465	465
query14	10298	9188	9229	9188
query15	216	208	190	190
query16	7187	717	521	521
query17	1183	755	619	619
query18	1988	417	328	328
query19	214	203	179	179
query20	127	126	121	121
query21	212	131	113	113
query22	3867	3923	3845	3845
query23	32856	32137	31890	31890
query24	8501	2396	2416	2396
query25	599	519	488	488
query26	1236	295	166	166
query27	2756	517	364	364
query28	4386	2147	2134	2134
query29	827	616	480	480
query30	310	243	216	216
query31	826	714	675	675
query32	82	71	75	71
query33	587	370	322	322
query34	791	875	523	523
query35	813	837	768	768
query36	903	949	859	859
query37	126	108	88	88
query38	3291	3426	3277	3277
query39	1481	1401	1402	1401
query40	227	137	121	121
query41	65	63	64	63
query42	131	119	117	117
query43	469	481	420	420
query44	1276	763	752	752
query45	202	198	183	183
query46	882	991	658	658
query47	1693	1704	1635	1635
query48	391	416	333	333
query49	783	493	399	399
query50	659	681	417	417
query51	3877	4022	4007	4007
query52	112	115	105	105
query53	243	268	191	191
query54	322	298	288	288
query55	88	90	88	88
query56	332	336	311	311
query57	1161	1134	1092	1092
query58	292	277	273	273
query59	2376	2445	2424	2424
query60	381	365	349	349
query61	195	185	184	184
query62	809	715	659	659
query63	237	205	203	203
query64	4703	1341	1042	1042
query65	4060	4029	3992	3992
query66	1189	465	350	350
query67	15386	14963	14707	14707
query68	8107	968	644	644
query69	520	353	333	333
query70	1331	1221	1177	1177
query71	455	345	324	324
query72	6042	4924	4862	4862
query73	646	597	375	375
query74	8665	8892	8422	8422
query75	3917	3315	2849	2849
query76	3544	1112	730	730
query77	805	426	308	308
query78	9705	9621	8992	8992
query79	2729	815	615	615
query80	697	590	503	503
query81	511	275	249	249
query82	482	164	141	141
query83	263	262	246	246
query84	272	123	101	101
query85	927	487	447	447
query86	386	312	290	290
query87	3582	3523	3435	3435
query88	4063	2198	2204	2198
query89	378	343	286	286
query90	1903	222	215	215
query91	175	173	138	138
query92	82	69	65	65
query93	2490	998	655	655
query94	685	474	342	342
query95	498	402	396	396
query96	498	555	291	291
query97	2927	2937	2870	2870
query98	238	208	203	203
query99	1339	1379	1271	1271
Total cold run time: 273817 ms
Total hot run time: 184320 ms

@doris-robot
Copy link

doris-robot commented Nov 26, 2025

ClickBench: Total hot run time: 27.51 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2abf36e577b257ff7bc8844418933390b3eca6da, data reload: false

query1	0.05	0.04	0.04
query2	0.09	0.06	0.05
query3	0.26	0.08	0.08
query4	1.61	0.11	0.11
query5	0.27	0.25	0.27
query6	1.18	0.65	0.64
query7	0.03	0.03	0.02
query8	0.06	0.04	0.04
query9	0.57	0.52	0.52
query10	0.58	0.56	0.56
query11	0.16	0.11	0.11
query12	0.16	0.12	0.12
query13	0.63	0.61	0.60
query14	1.00	1.00	1.00
query15	0.83	0.83	0.85
query16	0.38	0.38	0.39
query17	1.02	1.02	1.00
query18	0.21	0.20	0.19
query19	1.90	1.79	1.84
query20	0.02	0.01	0.01
query21	15.47	0.19	0.12
query22	5.01	0.06	0.05
query23	15.66	0.25	0.10
query24	2.28	0.86	0.46
query25	0.07	0.07	0.06
query26	0.15	0.13	0.13
query27	0.08	0.06	0.05
query28	4.17	1.15	0.95
query29	12.60	3.87	3.24
query30	0.28	0.14	0.12
query31	2.81	0.56	0.40
query32	3.23	0.55	0.47
query33	3.02	3.00	3.11
query34	15.80	5.21	4.60
query35	4.57	4.56	4.55
query36	0.68	0.52	0.50
query37	0.10	0.07	0.07
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.14	0.14
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 97.43 s
Total hot run time: 27.51 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 26, 2025

FE UT Coverage Report

Increment line coverage 69.23% (18/26) 🎉
Increment coverage report
Complete coverage report

@zddr
Copy link
Contributor Author

zddr commented Nov 26, 2025

run feut

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 26, 2025

FE Regression Coverage Report

Increment line coverage 80.77% (21/26) 🎉
Increment coverage report
Complete coverage report

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 26, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Nov 26, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 26, 2025

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit c1e233a into apache:master Nov 27, 2025
37 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 27, 2025
@zddr
[fix](meta)fix after the FE restarts, ConcurrentMap becomes LinkedHas…
6145704 
…hMap (#58382)

### What problem does this PR solve?

**For example, in the following case, after the FE restarts,
snapshotInfos will become a LinkedHashMap:**

@SerializedName("si")  
private Map<Long, SnapshotInfo> snapshotInfos = Maps.newConcurrentMap();

**The solution is to declare it as ConcurrentMap or, more specifically,
ConcurrentHashMap:**

@SerializedName("si")  
private ConcurrentHashMap<Long, SnapshotInfo> snapshotInfos =
Maps.newConcurrentMap();
@github-actions github-actions bot mentioned this pull request Nov 27, 2025
Merged
yiguolei pushed a commit that referenced this pull request Nov 27, 2025
@github-actions @zddr
branch-4.0: [fix](meta)fix after the FE restarts, ConcurrentMap becom…
efcd87d 
…es LinkedHashMap #58382 (#58446)

Cherry-picked from #58382

Co-authored-by: zhangdong <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@yiguolei yiguolei yiguolei approved these changes

@dataroaring dataroaring dataroaring approved these changes

@morrySnow morrySnow morrySnow approved these changes

@gavinchou gavinchou Awaiting requested review from gavinchou gavinchou is a code owner

@w41ter w41ter Awaiting requested review from w41ter w41ter is a code owner

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.2-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@zddr @hello-stephen @doris-robot @morningman @yiguolei @dataroaring @morrySnow