Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](Nereids) DatetimeV2 round floor was incorrectly implemented as round ceil #35153

Merged
merged 1 commit into from
May 22, 2024

Conversation

morrySnow
Copy link
Contributor

No description provided.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morrySnow
Copy link
Contributor Author

run buildall

@morrySnow morrySnow added usercase Important user case type label dev/2.0.x dev/2.1.x labels May 21, 2024
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 21, 2024
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 21, 2024
@morrySnow
Copy link
Contributor Author

run buildall

morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 21, 2024
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 21, 2024
@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40114 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 62929193cc6b5980e8cdb21938106dca397bdb3c, data reload: false

------ Round 1 ----------------------------------
q1	17600	4296	4339	4296
q2	2019	186	197	186
q3	10931	1290	1240	1240
q4	10552	798	890	798
q5	8072	2735	2646	2646
q6	220	134	137	134
q7	1015	580	583	580
q8	9257	2106	2074	2074
q9	9138	6686	6637	6637
q10	9378	3646	3693	3646
q11	447	246	243	243
q12	415	222	220	220
q13	18706	2978	2965	2965
q14	258	213	216	213
q15	523	480	464	464
q16	523	382	386	382
q17	970	628	719	628
q18	8061	7427	7409	7409
q19	2023	1518	1541	1518
q20	653	336	309	309
q21	5007	3238	4014	3238
q22	359	288	291	288
Total cold run time: 116127 ms
Total hot run time: 40114 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4409	4208	4240	4208
q2	365	275	267	267
q3	2969	2741	2699	2699
q4	1889	1619	1643	1619
q5	5278	5263	5264	5263
q6	213	124	124	124
q7	2225	1876	1855	1855
q8	3173	3352	3311	3311
q9	8334	8350	8344	8344
q10	3850	3656	3668	3656
q11	593	484	493	484
q12	750	591	602	591
q13	16666	2949	2943	2943
q14	304	257	259	257
q15	504	471	477	471
q16	470	405	426	405
q17	1751	1491	1466	1466
q18	7692	7633	7491	7491
q19	1639	1604	1565	1565
q20	1969	1781	1758	1758
q21	4896	4831	4836	4831
q22	587	528	519	519
Total cold run time: 70526 ms
Total hot run time: 54127 ms
@doris-robot
Copy link

TPC-DS: Total hot run time: 180670 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 62929193cc6b5980e8cdb21938106dca397bdb3c, data reload: false

query1	908	386	360	360
query2	6444	2435	2259	2259
query3	6656	216	219	216
query4	23210	21341	21482	21341
query5	4160	431	416	416
query6	267	172	197	172
query7	4584	288	292	288
query8	246	197	199	197
query9	8494	2375	2359	2359
query10	436	267	273	267
query11	14713	14149	14224	14149
query12	129	89	86	86
query13	1652	359	360	359
query14	10630	7161	8364	7161
query15	244	165	171	165
query16	8001	260	250	250
query17	1868	563	562	562
query18	2058	277	266	266
query19	192	146	145	145
query20	92	84	83	83
query21	197	126	123	123
query22	5117	4920	4929	4920
query23	34238	33292	33291	33291
query24	11612	2878	2843	2843
query25	650	367	360	360
query26	1773	170	153	153
query27	3013	318	328	318
query28	7264	2050	2006	2006
query29	1049	609	589	589
query30	311	177	177	177
query31	985	748	730	730
query32	91	51	52	51
query33	729	240	242	240
query34	1095	476	471	471
query35	870	681	643	643
query36	1074	885	937	885
query37	264	69	70	69
query38	2927	2776	2763	2763
query39	1627	1547	1554	1547
query40	270	121	127	121
query41	46	46	41	41
query42	104	102	96	96
query43	588	546	535	535
query44	1255	720	735	720
query45	256	257	245	245
query46	1073	724	722	722
query47	1996	1954	1908	1908
query48	363	286	289	286
query49	1181	391	408	391
query50	782	393	370	370
query51	6837	6744	6852	6744
query52	103	90	90	90
query53	351	288	288	288
query54	893	435	443	435
query55	78	75	75	75
query56	248	223	227	223
query57	1255	1139	1165	1139
query58	234	202	241	202
query59	3363	3089	3086	3086
query60	249	229	225	225
query61	90	87	88	87
query62	685	453	488	453
query63	309	278	282	278
query64	9781	2227	1712	1712
query65	3224	3085	3074	3074
query66	1363	345	357	345
query67	15372	15093	15123	15093
query68	4523	521	538	521
query69	474	298	298	298
query70	1204	1142	1141	1141
query71	395	265	260	260
query72	7173	2546	2386	2386
query73	711	317	314	314
query74	6654	6150	6229	6150
query75	3513	2593	2601	2593
query76	2886	996	915	915
query77	405	265	268	265
query78	10582	10083	10079	10079
query79	2495	503	511	503
query80	1086	424	427	424
query81	517	244	250	244
query82	1130	97	94	94
query83	239	168	166	166
query84	236	87	85	85
query85	1420	265	258	258
query86	448	330	306	306
query87	3341	3150	3045	3045
query88	4263	2375	2339	2339
query89	456	373	389	373
query90	1980	185	181	181
query91	122	95	96	95
query92	59	51	47	47
query93	1830	496	488	488
query94	1255	178	179	178
query95	391	301	297	297
query96	578	269	263	263
query97	3176	2995	3046	2995
query98	235	224	220	220
query99	1142	883	902	883
Total cold run time: 287138 ms
Total hot run time: 180670 ms
@doris-robot
Copy link

ClickBench: Total hot run time: 30.14 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 62929193cc6b5980e8cdb21938106dca397bdb3c, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.05
query3	0.23	0.06	0.05
query4	1.67	0.08	0.10
query5	0.51	0.50	0.49
query6	1.12	0.72	0.74
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.47
query10	0.54	0.54	0.55
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.60	0.58	0.61
query14	0.77	0.77	0.80
query15	0.82	0.82	0.81
query16	0.36	0.36	0.36
query17	0.95	0.98	0.96
query18	0.22	0.22	0.24
query19	1.76	1.66	1.66
query20	0.02	0.01	0.01
query21	15.46	0.73	0.69
query22	4.39	7.39	1.72
query23	18.31	1.47	1.20
query24	1.85	0.22	0.25
query25	0.15	0.09	0.07
query26	0.27	0.18	0.17
query27	0.07	0.08	0.09
query28	13.27	1.01	1.01
query29	12.63	3.29	3.23
query30	0.24	0.06	0.06
query31	2.90	0.38	0.38
query32	3.30	0.48	0.46
query33	2.84	2.88	2.96
query34	16.99	4.40	4.48
query35	4.49	4.56	4.46
query36	0.65	0.48	0.46
query37	0.18	0.15	0.16
query38	0.17	0.15	0.15
query39	0.04	0.03	0.04
query40	0.18	0.14	0.16
query41	0.09	0.04	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.17 s
Total hot run time: 30.14 s
Copy link
Contributor

@zclllyybb zclllyybb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by anyone and no changes requested.

morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 22, 2024
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 22, 2024
@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 42062 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bb8385f18a6a925f6f74e8d8eb5116a05b487dd8, data reload: false

------ Round 1 ----------------------------------
q1	17613	4964	4269	4269
q2	2021	192	195	192
q3	10469	1153	1142	1142
q4	10194	801	895	801
q5	7519	2724	2711	2711
q6	219	132	137	132
q7	1041	628	605	605
q8	9236	2202	2089	2089
q9	10053	7396	7443	7396
q10	9185	3893	3865	3865
q11	453	246	242	242
q12	452	233	230	230
q13	18206	3237	3171	3171
q14	260	223	214	214
q15	513	473	473	473
q16	520	416	419	416
q17	981	747	694	694
q18	8552	8035	7850	7850
q19	1673	1566	1561	1561
q20	652	320	322	320
q21	5316	4247	3390	3390
q22	351	299	307	299
Total cold run time: 115479 ms
Total hot run time: 42062 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4553	4430	4318	4318
q2	382	270	287	270
q3	3136	2879	2929	2879
q4	1912	1615	1600	1600
q5	5499	5534	5495	5495
q6	220	123	129	123
q7	2336	1985	1974	1974
q8	3269	3416	3391	3391
q9	9320	9495	9314	9314
q10	3879	3752	3853	3752
q11	591	519	513	513
q12	794	629	620	620
q13	17052	3167	3238	3167
q14	317	273	275	273
q15	525	485	475	475
q16	466	427	415	415
q17	1773	1491	1468	1468
q18	7773	7796	7518	7518
q19	6551	1607	1571	1571
q20	2014	1795	1781	1781
q21	5153	4758	4851	4758
q22	581	503	486	486
Total cold run time: 78096 ms
Total hot run time: 56161 ms
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 22, 2024
@doris-robot
Copy link

TPC-DS: Total hot run time: 181957 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bb8385f18a6a925f6f74e8d8eb5116a05b487dd8, data reload: false

query1	928	391	372	372
query2	6718	2545	2330	2330
query3	6652	210	210	210
query4	22856	21260	21371	21260
query5	4124	434	425	425
query6	270	181	180	180
query7	4581	306	294	294
query8	239	195	193	193
query9	8620	2415	2404	2404
query10	434	252	265	252
query11	14869	14261	14220	14220
query12	145	92	89	89
query13	1663	372	360	360
query14	10595	7783	8205	7783
query15	259	170	170	170
query16	8030	253	268	253
query17	1740	560	556	556
query18	2101	273	289	273
query19	199	150	156	150
query20	94	87	99	87
query21	196	136	129	129
query22	4947	4815	4852	4815
query23	34291	33682	33616	33616
query24	6885	2934	2983	2934
query25	575	387	365	365
query26	693	160	159	159
query27	2028	317	338	317
query28	3987	2072	2047	2047
query29	849	610	603	603
query30	268	168	177	168
query31	1003	767	751	751
query32	95	51	54	51
query33	472	250	244	244
query34	879	492	487	487
query35	759	683	668	668
query36	1094	904	955	904
query37	106	72	70	70
query38	2927	2758	2783	2758
query39	1613	1553	1555	1553
query40	195	135	128	128
query41	46	45	43	43
query42	107	92	105	92
query43	596	550	561	550
query44	1090	753	756	753
query45	279	249	249	249
query46	1059	721	725	721
query47	1943	1890	1898	1890
query48	364	299	286	286
query49	858	400	405	400
query50	777	380	392	380
query51	6862	6776	6624	6624
query52	102	93	90	90
query53	359	294	288	288
query54	548	434	442	434
query55	74	74	73	73
query56	238	227	218	218
query57	1224	1153	1172	1153
query58	214	211	195	195
query59	3470	3370	3129	3129
query60	281	234	237	234
query61	95	89	88	88
query62	617	485	477	477
query63	314	292	290	290
query64	8470	2204	1788	1788
query65	3133	3111	3080	3080
query66	785	361	335	335
query67	15428	14868	15062	14868
query68	4513	535	536	535
query69	472	306	309	306
query70	1106	1068	1135	1068
query71	369	272	274	272
query72	7021	2614	2355	2355
query73	705	324	324	324
query74	6570	6242	6132	6132
query75	3313	2618	2628	2618
query76	2262	984	968	968
query77	392	271	262	262
query78	10645	10197	10078	10078
query79	1682	525	521	521
query80	991	453	450	450
query81	531	243	237	237
query82	953	98	100	98
query83	249	169	170	169
query84	253	85	87	85
query85	981	279	280	279
query86	460	293	313	293
query87	3294	3146	3082	3082
query88	3082	2328	2334	2328
query89	458	391	397	391
query90	1990	191	196	191
query91	124	99	100	99
query92	66	49	48	48
query93	1671	516	499	499
query94	1041	186	186	186
query95	405	312	304	304
query96	586	268	277	268
query97	3178	3010	3034	3010
query98	241	224	217	217
query99	1182	929	928	928
Total cold run time: 269098 ms
Total hot run time: 181957 ms
@doris-robot
Copy link

ClickBench: Total hot run time: 31.01 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bb8385f18a6a925f6f74e8d8eb5116a05b487dd8, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.03	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.07
query5	0.49	0.49	0.51
query6	1.13	0.73	0.73
query7	0.02	0.01	0.01
query8	0.04	0.04	0.04
query9	0.54	0.49	0.49
query10	0.53	0.56	0.54
query11	0.16	0.11	0.12
query12	0.15	0.12	0.11
query13	0.60	0.59	0.61
query14	0.76	0.78	0.77
query15	0.83	0.83	0.81
query16	0.37	0.36	0.37
query17	1.00	1.01	0.98
query18	0.24	0.23	0.27
query19	1.86	1.78	1.75
query20	0.02	0.01	0.01
query21	15.72	0.65	0.64
query22	4.22	7.48	2.33
query23	18.29	1.39	1.23
query24	1.80	0.29	0.22
query25	0.15	0.08	0.08
query26	0.26	0.16	0.17
query27	0.07	0.08	0.07
query28	13.29	1.09	1.09
query29	13.03	3.25	3.25
query30	0.24	0.06	0.05
query31	2.87	0.40	0.38
query32	3.28	0.46	0.46
query33	2.87	2.90	2.92
query34	17.13	4.44	4.42
query35	4.50	4.73	4.50
query36	0.64	0.46	0.47
query37	0.17	0.15	0.16
query38	0.15	0.14	0.16
query39	0.05	0.04	0.03
query40	0.17	0.13	0.14
query41	0.09	0.04	0.05
query42	0.06	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.84 s
Total hot run time: 31.01 s
@morrySnow
Copy link
Contributor Author

run buildall

morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 22, 2024
@doris-robot
Copy link

TPC-DS: Total hot run time: 180923 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ec87be0533058cca52741836cb5d7f694aa905b8, data reload: false

query1	935	390	362	362
query2	6459	2294	2438	2294
query3	6655	206	205	205
query4	24730	21302	21312	21302
query5	4160	432	413	413
query6	266	174	177	174
query7	4599	291	285	285
query8	235	194	196	194
query9	8547	2395	2367	2367
query10	435	254	249	249
query11	14967	14162	14196	14162
query12	138	92	88	88
query13	1658	375	365	365
query14	10594	7088	7736	7088
query15	217	176	174	174
query16	7827	266	262	262
query17	1853	585	565	565
query18	1955	283	281	281
query19	209	151	155	151
query20	93	87	90	87
query21	199	133	132	132
query22	5158	4815	4829	4815
query23	34281	33464	33757	33464
query24	10840	2918	2801	2801
query25	637	365	362	362
query26	1427	155	153	153
query27	2965	327	327	327
query28	7373	2045	2034	2034
query29	922	638	599	599
query30	307	172	176	172
query31	1002	742	752	742
query32	88	51	51	51
query33	755	248	245	245
query34	1073	478	468	468
query35	814	676	656	656
query36	1093	926	906	906
query37	145	66	72	66
query38	2850	2765	2755	2755
query39	1608	1567	1532	1532
query40	195	126	125	125
query41	49	43	45	43
query42	106	93	97	93
query43	590	545	565	545
query44	1198	728	765	728
query45	259	246	242	242
query46	1066	731	712	712
query47	1936	1882	1880	1880
query48	355	303	294	294
query49	1081	392	389	389
query50	764	385	378	378
query51	6976	6723	6801	6723
query52	107	92	92	92
query53	347	283	294	283
query54	969	418	432	418
query55	76	71	74	71
query56	243	221	218	218
query57	1224	1137	1133	1133
query58	218	194	198	194
query59	3846	3064	3077	3064
query60	249	235	240	235
query61	89	87	92	87
query62	680	500	473	473
query63	307	285	287	285
query64	9889	2250	1722	1722
query65	3228	3093	3106	3093
query66	1446	349	340	340
query67	15412	15121	15216	15121
query68	4595	534	525	525
query69	485	296	298	296
query70	1207	1130	1151	1130
query71	386	269	264	264
query72	7560	2555	2379	2379
query73	703	316	316	316
query74	6659	6236	6185	6185
query75	3378	2677	2611	2611
query76	2825	959	1031	959
query77	439	264	262	262
query78	10778	10406	10088	10088
query79	2215	511	516	511
query80	1001	453	430	430
query81	549	248	245	245
query82	595	96	100	96
query83	248	163	170	163
query84	238	92	83	83
query85	1732	285	262	262
query86	504	318	305	305
query87	3275	3110	3081	3081
query88	3891	2336	2330	2330
query89	469	385	381	381
query90	1927	193	186	186
query91	125	102	100	100
query92	56	49	48	48
query93	1931	522	491	491
query94	1238	176	183	176
query95	391	299	291	291
query96	575	266	265	265
query97	3166	3048	3002	3002
query98	244	227	225	225
query99	1144	914	908	908
Total cold run time: 287503 ms
Total hot run time: 180923 ms
@doris-robot
Copy link

ClickBench: Total hot run time: 30.63 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ec87be0533058cca52741836cb5d7f694aa905b8, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.03
query3	0.22	0.05	0.06
query4	1.67	0.09	0.11
query5	0.49	0.49	0.51
query6	1.13	0.74	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.53	0.48	0.48
query10	0.54	0.55	0.54
query11	0.15	0.12	0.11
query12	0.14	0.13	0.12
query13	0.59	0.59	0.60
query14	0.78	0.77	0.79
query15	0.82	0.80	0.82
query16	0.36	0.36	0.34
query17	0.96	0.96	0.92
query18	0.25	0.23	0.23
query19	1.73	1.67	1.66
query20	0.01	0.02	0.02
query21	15.43	0.71	0.68
query22	4.38	6.29	2.25
query23	18.29	1.39	1.30
query24	1.83	0.27	0.20
query25	0.16	0.08	0.07
query26	0.28	0.16	0.17
query27	0.08	0.07	0.08
query28	13.28	1.01	1.00
query29	12.63	3.28	3.23
query30	0.25	0.06	0.05
query31	2.85	0.38	0.38
query32	3.30	0.47	0.48
query33	2.89	2.92	2.85
query34	17.15	4.40	4.46
query35	4.46	4.48	4.44
query36	0.65	0.46	0.46
query37	0.18	0.15	0.15
query38	0.15	0.15	0.14
query39	0.04	0.04	0.03
query40	0.16	0.15	0.14
query41	0.09	0.04	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.19 s
Total hot run time: 30.63 s
@morrySnow
Copy link
Contributor Author

run p0

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 22, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@morrySnow morrySnow merged commit e4f41db into apache:master May 22, 2024
26 of 28 checks passed
@morrySnow morrySnow deleted the fix_datetimev2_round_floor branch May 22, 2024 08:21
morrySnow added a commit that referenced this pull request May 22, 2024
) (#35154)

pick from master #35153

1.  round floor was incorrectly implemented as round
2. round ceiling not really round because use double type when divide
morrySnow added a commit that referenced this pull request May 22, 2024
) (#35155)

pick from master #35153

1.  round floor was incorrectly implemented as round
2. round ceiling not really round because use double type when divide
dataroaring pushed a commit that referenced this pull request May 26, 2024
)

1.  round floor was incorrectly implemented as round
2. round ceiling not really round because use double type when divide
@morningman morningman mentioned this pull request Jun 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.11-merged dev/2.1.4-merged reviewed usercase Important user case type label
5 participants