Skip to content

Commit c7f39df

Browse files
gene-bordegaraygene-bordegaray
committed
Fixed benchmark
1 parent 288817b commit c7f39df

File tree

15 files changed

+339
-400
lines changed

15 files changed

+339
-400
lines changed

‎datafusion/core/tests/physical_optimizer/enforce_distribution.rs‎

Lines changed: 74 additions & 100 deletions
Large diffs are not rendered by default.

‎datafusion/sqllogictest/test_files/tpch/plans/q10.slt.part‎

Lines changed: 15 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -87,20 +87,18 @@ physical_plan
8787
16)------------------------------CoalesceBatchesExec: target_batch_size=8192
8888
17)--------------------------------HashJoinExec: mode=Partitioned, join_type=Inner, on=[(c_custkey@0, o_custkey@1)], projection=[c_custkey@0, c_name@1, c_address@2, c_nationkey@3, c_phone@4, c_acctbal@5, c_comment@6, o_orderkey@7]
8989
18)----------------------------------CoalesceBatchesExec: target_batch_size=8192
90-
19)------------------------------------RepartitionExec: partitioning=Hash([c_custkey@0], 4), input_partitions=4
91-
20)--------------------------------------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
92-
21)----------------------------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/customer.tbl]]}, projection=[c_custkey, c_name, c_address, c_nationkey, c_phone, c_acctbal, c_comment], file_type=csv, has_header=false
93-
22)----------------------------------CoalesceBatchesExec: target_batch_size=8192
94-
23)------------------------------------RepartitionExec: partitioning=Hash([o_custkey@1], 4), input_partitions=4
95-
24)--------------------------------------CoalesceBatchesExec: target_batch_size=8192
96-
25)----------------------------------------FilterExec: o_orderdate@2 >= 1993-10-01 AND o_orderdate@2 < 1994-01-01, projection=[o_orderkey@0, o_custkey@1]
97-
26)------------------------------------------DataSourceExec: file_groups={4 groups: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:0..4223281], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:4223281..8446562], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:8446562..12669843], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:12669843..16893122]]}, projection=[o_orderkey, o_custkey, o_orderdate], file_type=csv, has_header=false
98-
27)--------------------------CoalesceBatchesExec: target_batch_size=8192
99-
28)----------------------------RepartitionExec: partitioning=Hash([l_orderkey@0], 4), input_partitions=4
100-
29)------------------------------CoalesceBatchesExec: target_batch_size=8192
101-
30)--------------------------------FilterExec: l_returnflag@3 = R, projection=[l_orderkey@0, l_extendedprice@1, l_discount@2]
102-
31)----------------------------------DataSourceExec: file_groups={4 groups: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:0..18561749], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:18561749..37123498], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:37123498..55685247], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:55685247..74246996]]}, projection=[l_orderkey, l_extendedprice, l_discount, l_returnflag], file_type=csv, has_header=false
103-
32)------------------CoalesceBatchesExec: target_batch_size=8192
104-
33)--------------------RepartitionExec: partitioning=Hash([n_nationkey@0], 4), input_partitions=4
105-
34)----------------------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
106-
35)------------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/nation.tbl]]}, projection=[n_nationkey, n_name], file_type=csv, has_header=false
90+
19)------------------------------------RepartitionExec: partitioning=Hash([c_custkey@0], 4), input_partitions=1
91+
20)--------------------------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/customer.tbl]]}, projection=[c_custkey, c_name, c_address, c_nationkey, c_phone, c_acctbal, c_comment], file_type=csv, has_header=false
92+
21)----------------------------------CoalesceBatchesExec: target_batch_size=8192
93+
22)------------------------------------RepartitionExec: partitioning=Hash([o_custkey@1], 4), input_partitions=4
94+
23)--------------------------------------CoalesceBatchesExec: target_batch_size=8192
95+
24)----------------------------------------FilterExec: o_orderdate@2 >= 1993-10-01 AND o_orderdate@2 < 1994-01-01, projection=[o_orderkey@0, o_custkey@1]
96+
25)------------------------------------------DataSourceExec: file_groups={4 groups: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:0..4223281], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:4223281..8446562], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:8446562..12669843], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:12669843..16893122]]}, projection=[o_orderkey, o_custkey, o_orderdate], file_type=csv, has_header=false
97+
26)--------------------------CoalesceBatchesExec: target_batch_size=8192
98+
27)----------------------------RepartitionExec: partitioning=Hash([l_orderkey@0], 4), input_partitions=4
99+
28)------------------------------CoalesceBatchesExec: target_batch_size=8192
100+
29)--------------------------------FilterExec: l_returnflag@3 = R, projection=[l_orderkey@0, l_extendedprice@1, l_discount@2]
101+
30)----------------------------------DataSourceExec: file_groups={4 groups: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:0..18561749], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:18561749..37123498], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:37123498..55685247], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:55685247..74246996]]}, projection=[l_orderkey, l_extendedprice, l_discount, l_returnflag], file_type=csv, has_header=false
102+
31)------------------CoalesceBatchesExec: target_batch_size=8192
103+
32)--------------------RepartitionExec: partitioning=Hash([n_nationkey@0], 4), input_partitions=1
104+
33)----------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/nation.tbl]]}, projection=[n_nationkey, n_name], file_type=csv, has_header=false

‎datafusion/sqllogictest/test_files/tpch/plans/q11.slt.part‎

Lines changed: 30 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -92,35 +92,33 @@ physical_plan
9292
17)--------------------------------RepartitionExec: partitioning=Hash([ps_suppkey@1], 4), input_partitions=4
9393
18)----------------------------------DataSourceExec: file_groups={4 groups: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/partsupp.tbl:0..2932049], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/partsupp.tbl:2932049..5864098], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/partsupp.tbl:5864098..8796147], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/partsupp.tbl:8796147..11728193]]}, projection=[ps_partkey, ps_suppkey, ps_availqty, ps_supplycost], file_type=csv, has_header=false
9494
19)------------------------------CoalesceBatchesExec: target_batch_size=8192
95-
20)--------------------------------RepartitionExec: partitioning=Hash([s_suppkey@0], 4), input_partitions=4
96-
21)----------------------------------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
97-
22)------------------------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/supplier.tbl]]}, projection=[s_suppkey, s_nationkey], file_type=csv, has_header=false
98-
23)----------------------CoalesceBatchesExec: target_batch_size=8192
99-
24)------------------------RepartitionExec: partitioning=Hash([n_nationkey@0], 4), input_partitions=4
100-
25)--------------------------CoalesceBatchesExec: target_batch_size=8192
101-
26)----------------------------FilterExec: n_name@1 = GERMANY, projection=[n_nationkey@0]
102-
27)------------------------------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
103-
28)--------------------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/nation.tbl]]}, projection=[n_nationkey, n_name], file_type=csv, has_header=false
104-
29)------ProjectionExec: expr=[CAST(CAST(sum(partsupp.ps_supplycost * partsupp.ps_availqty)@0 AS Float64) * 0.0001 AS Decimal128(38, 15)) as sum(partsupp.ps_supplycost * partsupp.ps_availqty) * Float64(0.0001)]
105-
30)--------AggregateExec: mode=Final, gby=[], aggr=[sum(partsupp.ps_supplycost * partsupp.ps_availqty)]
106-
31)----------CoalescePartitionsExec
107-
32)------------AggregateExec: mode=Partial, gby=[], aggr=[sum(partsupp.ps_supplycost * partsupp.ps_availqty)]
108-
33)--------------CoalesceBatchesExec: target_batch_size=8192
109-
34)----------------HashJoinExec: mode=Partitioned, join_type=Inner, on=[(s_nationkey@2, n_nationkey@0)], projection=[ps_availqty@0, ps_supplycost@1]
110-
35)------------------CoalesceBatchesExec: target_batch_size=8192
111-
36)--------------------RepartitionExec: partitioning=Hash([s_nationkey@2], 4), input_partitions=4
112-
37)----------------------CoalesceBatchesExec: target_batch_size=8192
113-
38)------------------------HashJoinExec: mode=Partitioned, join_type=Inner, on=[(ps_suppkey@0, s_suppkey@0)], projection=[ps_availqty@1, ps_supplycost@2, s_nationkey@4]
114-
39)--------------------------CoalesceBatchesExec: target_batch_size=8192
115-
40)----------------------------RepartitionExec: partitioning=Hash([ps_suppkey@0], 4), input_partitions=4
116-
41)------------------------------DataSourceExec: file_groups={4 groups: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/partsupp.tbl:0..2932049], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/partsupp.tbl:2932049..5864098], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/partsupp.tbl:5864098..8796147], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/partsupp.tbl:8796147..11728193]]}, projection=[ps_suppkey, ps_availqty, ps_supplycost], file_type=csv, has_header=false
117-
42)--------------------------CoalesceBatchesExec: target_batch_size=8192
118-
43)----------------------------RepartitionExec: partitioning=Hash([s_suppkey@0], 4), input_partitions=4
119-
44)------------------------------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
120-
45)--------------------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/supplier.tbl]]}, projection=[s_suppkey, s_nationkey], file_type=csv, has_header=false
121-
46)------------------CoalesceBatchesExec: target_batch_size=8192
122-
47)--------------------RepartitionExec: partitioning=Hash([n_nationkey@0], 4), input_partitions=4
123-
48)----------------------CoalesceBatchesExec: target_batch_size=8192
124-
49)------------------------FilterExec: n_name@1 = GERMANY, projection=[n_nationkey@0]
125-
50)--------------------------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
126-
51)----------------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/nation.tbl]]}, projection=[n_nationkey, n_name], file_type=csv, has_header=false
95+
20)--------------------------------RepartitionExec: partitioning=Hash([s_suppkey@0], 4), input_partitions=1
96+
21)----------------------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/supplier.tbl]]}, projection=[s_suppkey, s_nationkey], file_type=csv, has_header=false
97+
22)----------------------CoalesceBatchesExec: target_batch_size=8192
98+
23)------------------------RepartitionExec: partitioning=Hash([n_nationkey@0], 4), input_partitions=4
99+
24)--------------------------CoalesceBatchesExec: target_batch_size=8192
100+
25)----------------------------FilterExec: n_name@1 = GERMANY, projection=[n_nationkey@0]
101+
26)------------------------------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
102+
27)--------------------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/nation.tbl]]}, projection=[n_nationkey, n_name], file_type=csv, has_header=false
103+
28)------ProjectionExec: expr=[CAST(CAST(sum(partsupp.ps_supplycost * partsupp.ps_availqty)@0 AS Float64) * 0.0001 AS Decimal128(38, 15)) as sum(partsupp.ps_supplycost * partsupp.ps_availqty) * Float64(0.0001)]
104+
29)--------AggregateExec: mode=Final, gby=[], aggr=[sum(partsupp.ps_supplycost * partsupp.ps_availqty)]
105+
30)----------CoalescePartitionsExec
106+
31)------------AggregateExec: mode=Partial, gby=[], aggr=[sum(partsupp.ps_supplycost * partsupp.ps_availqty)]
107+
32)--------------CoalesceBatchesExec: target_batch_size=8192
108+
33)----------------HashJoinExec: mode=Partitioned, join_type=Inner, on=[(s_nationkey@2, n_nationkey@0)], projection=[ps_availqty@0, ps_supplycost@1]
109+
34)------------------CoalesceBatchesExec: target_batch_size=8192
110+
35)--------------------RepartitionExec: partitioning=Hash([s_nationkey@2], 4), input_partitions=4
111+
36)----------------------CoalesceBatchesExec: target_batch_size=8192
112+
37)------------------------HashJoinExec: mode=Partitioned, join_type=Inner, on=[(ps_suppkey@0, s_suppkey@0)], projection=[ps_availqty@1, ps_supplycost@2, s_nationkey@4]
113+
38)--------------------------CoalesceBatchesExec: target_batch_size=8192
114+
39)----------------------------RepartitionExec: partitioning=Hash([ps_suppkey@0], 4), input_partitions=4
115+
40)------------------------------DataSourceExec: file_groups={4 groups: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/partsupp.tbl:0..2932049], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/partsupp.tbl:2932049..5864098], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/partsupp.tbl:5864098..8796147], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/partsupp.tbl:8796147..11728193]]}, projection=[ps_suppkey, ps_availqty, ps_supplycost], file_type=csv, has_header=false
116+
41)--------------------------CoalesceBatchesExec: target_batch_size=8192
117+
42)----------------------------RepartitionExec: partitioning=Hash([s_suppkey@0], 4), input_partitions=1
118+
43)------------------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/supplier.tbl]]}, projection=[s_suppkey, s_nationkey], file_type=csv, has_header=false
119+
44)------------------CoalesceBatchesExec: target_batch_size=8192
120+
45)--------------------RepartitionExec: partitioning=Hash([n_nationkey@0], 4), input_partitions=4
121+
46)----------------------CoalesceBatchesExec: target_batch_size=8192
122+
47)------------------------FilterExec: n_name@1 = GERMANY, projection=[n_nationkey@0]
123+
48)--------------------------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
124+
49)----------------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/nation.tbl]]}, projection=[n_nationkey, n_name], file_type=csv, has_header=false

‎datafusion/sqllogictest/test_files/tpch/plans/q13.slt.part‎

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -65,11 +65,10 @@ physical_plan
6565
10)------------------CoalesceBatchesExec: target_batch_size=8192
6666
11)--------------------HashJoinExec: mode=Partitioned, join_type=Left, on=[(c_custkey@0, o_custkey@1)], projection=[c_custkey@0, o_orderkey@1]
6767
12)----------------------CoalesceBatchesExec: target_batch_size=8192
68-
13)------------------------RepartitionExec: partitioning=Hash([c_custkey@0], 4), input_partitions=4
69-
14)--------------------------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
70-
15)----------------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/customer.tbl]]}, projection=[c_custkey], file_type=csv, has_header=false
71-
16)----------------------CoalesceBatchesExec: target_batch_size=8192
72-
17)------------------------RepartitionExec: partitioning=Hash([o_custkey@1], 4), input_partitions=4
73-
18)--------------------------CoalesceBatchesExec: target_batch_size=8192
74-
19)----------------------------FilterExec: o_comment@2 NOT LIKE %special%requests%, projection=[o_orderkey@0, o_custkey@1]
75-
20)------------------------------DataSourceExec: file_groups={4 groups: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:0..4223281], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:4223281..8446562], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:8446562..12669843], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:12669843..16893122]]}, projection=[o_orderkey, o_custkey, o_comment], file_type=csv, has_header=false
68+
13)------------------------RepartitionExec: partitioning=Hash([c_custkey@0], 4), input_partitions=1
69+
14)--------------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/customer.tbl]]}, projection=[c_custkey], file_type=csv, has_header=false
70+
15)----------------------CoalesceBatchesExec: target_batch_size=8192
71+
16)------------------------RepartitionExec: partitioning=Hash([o_custkey@1], 4), input_partitions=4
72+
17)--------------------------CoalesceBatchesExec: target_batch_size=8192
73+
18)----------------------------FilterExec: o_comment@2 NOT LIKE %special%requests%, projection=[o_orderkey@0, o_custkey@1]
74+
19)------------------------------DataSourceExec: file_groups={4 groups: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:0..4223281], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:4223281..8446562], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:8446562..12669843], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/orders.tbl:12669843..16893122]]}, projection=[o_orderkey, o_custkey, o_comment], file_type=csv, has_header=false

‎datafusion/sqllogictest/test_files/tpch/plans/q14.slt.part‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,5 @@ physical_plan
5454
11)--------------------FilterExec: l_shipdate@3 >= 1995-09-01 AND l_shipdate@3 < 1995-10-01, projection=[l_partkey@0, l_extendedprice@1, l_discount@2]
5555
12)----------------------DataSourceExec: file_groups={4 groups: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:0..18561749], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:18561749..37123498], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:37123498..55685247], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:55685247..74246996]]}, projection=[l_partkey, l_extendedprice, l_discount, l_shipdate], file_type=csv, has_header=false
5656
13)--------------CoalesceBatchesExec: target_batch_size=8192
57-
14)----------------RepartitionExec: partitioning=Hash([p_partkey@0], 4), input_partitions=4
58-
15)------------------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
59-
16)--------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/part.tbl]]}, projection=[p_partkey, p_type], file_type=csv, has_header=false
57+
14)----------------RepartitionExec: partitioning=Hash([p_partkey@0], 4), input_partitions=1
58+
15)------------------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/part.tbl]]}, projection=[p_partkey, p_type], file_type=csv, has_header=false

0 commit comments

Comments
 (0)