crp-order-quantity-kpi-service - Details for Query 72386

Submitted Time: 2026/05/22 09:31:38
Duration: 2 s
Succeeded Jobs: 115307 115308 115309 115310 115311 115312 115313

Plan Visualization

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [id="node0" labelType="html" label=" AdaptiveSparkPlan " tooltip="AdaptiveSparkPlan isFinalPlan=true"]; 1 [id="node1" labelType="html" label=" EmptyRelation " tooltip="EmptyRelation [plan_id=21682909]"]; 2 [id="node2" labelType="html" label=" GlobalLimit " tooltip="GlobalLimit 1"]; 3 [id="node3" labelType="html" label=" EmptyRelation " tooltip="EmptyRelation LocalLimit 1"]; 4 [id="node4" labelType="html" label=" LocalLimit " tooltip="LocalLimit 1"]; 5 [id="node5" labelType="html" label=" LogicalQueryStage " tooltip="LogicalQueryStage Project, TableCacheQueryStage 0"]; 6 [id="node6" labelType="html" label=" InMemoryTableScan " tooltip="InMemoryTableScan"]; 7 [id="node7" labelType="html" label=" AdaptiveSparkPlan " tooltip="AdaptiveSparkPlan isFinalPlan=true"]; subgraph cluster8 { isCluster="true"; id="cluster8"; label="WholeStageCodegen (7)\n \nduration: total (min, med, max (stageId: taskId))\n3.6 s (0 ms, 1 ms, 371 ms (stage 316341.0: task 4666422))"; tooltip="WholeStageCodegen (7)"; 9 [id="node9" labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 214 ms (0 ms, 0 ms, 19 ms (stage 316341.0: task 4666397)) peak memory total (min, med, max (stageId: taskId)) 9.3 MiB (256.0 KiB, 256.0 KiB, 256.0 KiB (stage 316341.0: task 4666405)) number of output rows: 0 number of sort fallback tasks: 0 avg hash probes per key: 0" tooltip="HashAggregate(keys=[product#15459015, plant#15459016], functions=[])"]; 10 [id="node10" labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 117 ms (0 ms, 0 ms, 13 ms (stage 316341.0: task 4666401)) peak memory total (min, med, max (stageId: taskId)) 9.3 MiB (256.0 KiB, 256.0 KiB, 256.0 KiB (stage 316341.0: task 4666405)) number of output rows: 0 number of sort fallback tasks: 0 avg hash probes per key: 0" tooltip="HashAggregate(keys=[product#15459015, plant#15459016], functions=[])"]; 11 [id="node11" labelType="html" label=" Project " tooltip="Project [product#15459015, plant#15459016]"]; 12 [id="node12" labelType="html" label="Filter number of output rows: 0" tooltip="Filter (((size(demandTimeBuckets#15459407, true) = 0) OR isnull(demandTimeBuckets#15459407)) AND ((size(demandPointInTime#15459410, true) = 0) OR isnull(demandPointInTime#15459410)))"]; 13 [id="node13" labelType="html" label="BroadcastHashJoin number of output rows: 8" tooltip="BroadcastHashJoin [coalesce(plant#15459016, ), isnull(plant#15459016), coalesce(product#15459015, ), isnull(product#15459015)], [coalesce(plant#15459403, ), isnull(plant#15459403), coalesce(product#15459402, ), isnull(product#15459402)], LeftOuter, BuildRight, false"]; 14 [id="node14" labelType="html" label=" Project " tooltip="Project [product#15459015, plant#15459016]"]; 15 [id="node15" labelType="html" label="Generate number of output rows: 8" tooltip="Generate explode(_extract_quantityDenominator#15459437), [product#15459015, plant#15459016], true, [quantityDenominator#15459057]"]; 16 [id="node16" labelType="html" label=" Project " tooltip="Project [product#15459015, plant#15459016, filteredUnitOfMeasureConversions#15459046.quantityDenominator AS _extract_quantityDenominator#15459437]"]; 17 [id="node17" labelType="html" label="Generate number of output rows: 8" tooltip="Generate explode(filteredUnitOfMeasureConversions#15459046.quantityNumerator), [product#15459015, plant#15459016, filteredUnitOfMeasureConversions#15459046], true, [quantityNumerator#15459054]"]; } 18 [id="node18" labelType="html" label=" Project " tooltip="Project [product#15459015, plant#15459016, filter(unitOfMeasureConversions#15459040, lambdafunction((((lambda u#15459047.measurementUnit1.code = _extract_code#15459518) AND (lambda u#15459047.measurementUnit1.internalRefUUID = _extract_internalRefUUID#15459519)) AND ((lambda u#15459047.measurementUnit2.code = baseUnitOfMeasure#15459038.code) AND (lambda u#15459047.measurementUnit2.internalRefUUID = baseUnitOfMeasure#15459038.internalRefUUID))), lambda u#15459047, false)) AS filteredUnitOfMeasureConversions#15459046]"]; subgraph cluster19 { isCluster="true"; id="cluster19"; label="WholeStageCodegen (6)\n \nduration: total (min, med, max (stageId: taskId))\n7.7 s (1 ms, 2 ms, 447 ms (stage 316341.0: task 4666401))"; tooltip="WholeStageCodegen (6)"; 20 [id="node20" labelType="html" label="BroadcastHashJoin number of output rows: 8" tooltip="BroadcastHashJoin [product#15459015], [internalUUID#15459034], LeftOuter, BuildRight, false"]; 21 [id="node21" labelType="html" label=" Project " tooltip="Project [product#15459015, plant#15459016, _extract_code#15459518, _extract_internalRefUUID#15459519]"]; 22 [id="node22" labelType="html" label="BroadcastHashJoin number of output rows: 8" tooltip="BroadcastHashJoin [coalesce(product#15459015, ), isnull(product#15459015), coalesce(plant#15459016, ), isnull(plant#15459016)], [coalesce(_extract_internalRefUUID#15459517, ), isnull(_extract_internalRefUUID#15459517), coalesce(_extract_internalRefUUID#15459516, ), isnull(_extract_internalRefUUID#15459516)], LeftOuter, BuildRight, false"]; } 23 [id="node23" labelType="html" label="Exchange shuffle records written: 8 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 8 local bytes read total (min, med, max (stageId: taskId)) 1152.0 B (144.0 B, 144.0 B, 144.0 B (stage 316341.0: task 4666397)) merged fetch fallback count: 0 local blocks read: 8 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 704.0 B (88.0 B, 88.0 B, 88.0 B (stage 316332.0: task 4666316)) local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 4 ms (0 ms, 0 ms, 0 ms (stage 316332.0: task 4666317)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 316341.0: task 4666397)) remote bytes read: 0.0 B number of partitions: 37 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 1152.0 B (144.0 B, 144.0 B, 144.0 B (stage 316332.0: task 4666316))" tooltip="Exchange hashpartitioning(product#15459015, plant#15459016, 37), REPARTITION_BY_NUM, [plan_id=21682439]"]; subgraph cluster24 { isCluster="true"; id="cluster24"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n2.1 s (191 ms, 222 ms, 334 ms (stage 316332.0: task 4666313))"; tooltip="WholeStageCodegen (1)"; 25 [id="node25" labelType="html" label="ColumnarToRow number of output rows: 8 number of input batches: 8" tooltip="ColumnarToRow"]; } 26 [id="node26" labelType="html" label="Scan parquet number of files read: 9 scan time total (min, med, max (stageId: taskId)) 1.2 s (92 ms, 122 ms, 244 ms (stage 316332.0: task 4666313)) metadata time: 0 ms size of files read: 14.6 KiB number of output rows: 8" tooltip="FileScan parquet [product#15459015,plant#15459016] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[hdlfs://332d7051-0d16-44f5-ae76-bf2dcf30365f.files.hdl.prod-eu20.hanac..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<product:string,plant:string>"]; 27 [id="node27" labelType="html" label="BroadcastExchange time to broadcast: 1 ms time to build: 1 ms time to collect: 115 ms number of output rows: 1,596 data size: 2.1 MiB" tooltip="BroadcastExchange HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=21682732]"]; subgraph cluster28 { isCluster="true"; id="cluster28"; label="WholeStageCodegen (5)\n \nduration: total (min, med, max (stageId: taskId))\n10 ms (0 ms, 0 ms, 2 ms (stage 316339.0: task 4666362))"; tooltip="WholeStageCodegen (5)"; 29 [id="node29" labelType="html" label=" Project " tooltip="Project [_extract_code#15459518, _extract_internalRefUUID#15459519, _extract_internalRefUUID#15459517, _extract_internalRefUUID#15459516]"]; } 30 [id="node30" labelType="html" label="Exchange shuffle records written: 1,596 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1,596 local bytes read total (min, med, max (stageId: taskId)) 86.4 KiB (2.1 KiB, 2.3 KiB, 2.5 KiB (stage 316339.0: task 4666395)) merged fetch fallback count: 0 local blocks read: 37 remote merged chunks fetched: 0 remote blocks read: 0 data size: 215.5 KiB local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 316339.0: task 4666362)) remote bytes read: 0.0 B number of partitions: 37 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 86.4 KiB" tooltip="Exchange RoundRobinPartitioning(37), REPARTITION_BY_NUM, [plan_id=21682460]"]; subgraph cluster31 { isCluster="true"; id="cluster31"; label="WholeStageCodegen (2)\n \nduration: 163 ms"; tooltip="WholeStageCodegen (2)"; 32 [id="node32" labelType="html" label=" Project " tooltip="Project [plant#15459022.internalRefUUID AS _extract_internalRefUUID#15459516, product#15459023.internalRefUUID AS _extract_internalRefUUID#15459517, productionAspect#15459025.productMovementPlants.unitOfIssue.code AS _extract_code#15459518, productionAspect#15459025.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#15459519]"]; 33 [id="node33" labelType="html" label="ColumnarToRow number of output rows: 1,596 number of input batches: 1" tooltip="ColumnarToRow"]; } 34 [id="node34" labelType="html" label="Scan parquet number of files read: 1 scan time: 147 ms dynamic partition pruning time: 0 ms metadata time: 0 ms size of files read: 91.6 KiB number of output rows: 1,596 number of partitions read: 1" tooltip="FileScan parquet [plant#15459022,product#15459023,productionAspect#15459025,systemId#15459020] Batched: true, DataFilters: [], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[hdlfs://332d7051-0d16-44f5-ae76-bf2dcf30365f.files.hdl.prod-eu20...., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspe..."]; 35 [id="node35" labelType="html" label="BroadcastExchange time to broadcast: 0 ms time to build: 3 ms time to collect: 54 ms number of output rows: 150 data size: 2.0 MiB" tooltip="BroadcastExchange HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=21682498]"]; 36 [id="node36" labelType="html" label="Exchange shuffle records written: 150 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 150 local bytes read total (min, med, max (stageId: taskId)) 28.4 KiB (535.0 B, 772.0 B, 1068.0 B (stage 316337.0: task 4666322)) merged fetch fallback count: 0 local blocks read: 37 remote merged chunks fetched: 0 remote blocks read: 0 data size: 89.7 KiB local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 316337.0: task 4666325)) remote bytes read: 0.0 B number of partitions: 37 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 28.4 KiB" tooltip="Exchange RoundRobinPartitioning(37), REPARTITION_BY_NUM, [plan_id=21682488]"]; subgraph cluster37 { isCluster="true"; id="cluster37"; label="WholeStageCodegen (3)\n \nduration: 154 ms"; tooltip="WholeStageCodegen (3)"; 38 [id="node38" labelType="html" label=" Project " tooltip="Project [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040]"]; 39 [id="node39" labelType="html" label="Filter number of output rows: 150" tooltip="Filter isnotnull(internalUUID#15459034)"]; 40 [id="node40" labelType="html" label="ColumnarToRow number of output rows: 150 number of input batches: 1" tooltip="ColumnarToRow"]; } 41 [id="node41" labelType="html" label="Scan parquet number of files read: 1 scan time: 146 ms dynamic partition pruning time: 0 ms metadata time: 0 ms size of files read: 34.7 KiB number of output rows: 150 number of partitions read: 1" tooltip="FileScan parquet [internalUUID#15459034,baseUnitOfMeasure#15459038,unitOfMeasureConversions#15459040,systemId#15459033] Batched: true, DataFilters: [isnotnull(internalUUID#15459034)], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[hdlfs://332d7051-0d16-44f5-ae76-bf2dcf30365f.files.hdl.prod-eu20...., PartitionFilters: [], PushedFilters: [IsNotNull(internalUUID)], ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMea..."]; 42 [id="node42" labelType="html" label="BroadcastExchange time to broadcast: 0 ms time to build: 4 ms time to collect: 371 ms number of output rows: 8 data size: 2.0 MiB" tooltip="BroadcastExchange HashedRelationBroadcastMode(List(coalesce(input[1, string, true], ), isnull(input[1, string, true]), coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=21682512]"]; subgraph cluster43 { isCluster="true"; id="cluster43"; label="WholeStageCodegen (4)\n \nduration: 136 ms"; tooltip="WholeStageCodegen (4)"; 44 [id="node44" labelType="html" label="ColumnarToRow number of output rows: 8 number of input batches: 1" tooltip="ColumnarToRow"]; } 45 [id="node45" labelType="html" label="Scan parquet number of files read: 1 scan time: 121 ms metadata time: 0 ms size of files read: 9.8 KiB number of output rows: 8" tooltip="FileScan parquet [product#15459402,plant#15459403,demandTimeBuckets#15459407,demandPointInTime#15459410] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[hdlfs://332d7051-0d16-44f5-ae76-bf2dcf30365f.files.hdl.prod-eu20.hanac..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<product:string,plant:string,demandTimeBuckets:array<struct<demandStartDateTime:timestamp,d..."]; 1->0; 2->1; 3->2; 4->3; 5->4; 6->5; 7->6; 9->7; 10->9; 11->10; 12->11; 13->12; 14->13; 15->14; 16->15; 17->16; 18->17; 20->18; 21->20; 22->21; 23->22; 25->23; 26->25; 27->22; 29->27; 30->29; 32->30; 33->32; 34->33; 35->20; 36->35; 38->36; 39->38; 40->39; 41->40; 42->13; 44->42; 45->44; }

Details

== Physical Plan ==
AdaptiveSparkPlan (68)
+- == Final Plan ==
   ResultQueryStage (2), Statistics(sizeInBytes=8.0 EiB)
   +- EmptyRelation (1)
      +- GlobalLimit (unknown)
         +- EmptyRelation (unknown)
+- == Initial Plan ==
   CollectLimit (67)
   +- InMemoryTableScan (3)
         +- InMemoryRelation (4)
               +- AdaptiveSparkPlan (66)
               +- == Final Plan ==
                  ResultQueryStage (42)
                  +- * HashAggregate (41)
                     +- * HashAggregate (40)
                        +- * Project (39)
                           +- * Filter (38)
                              +- * BroadcastHashJoin LeftOuter BuildRight (37)
                                 :- * Project (32)
                                 :  +- * Generate (31)
                                 :     +- * Project (30)
                                 :        +- * Generate (29)
                                 :           +- Project (28)
                                 :              +- * BroadcastHashJoin LeftOuter BuildRight (27)
                                 :                 :- * Project (18)
                                 :                 :  +- * BroadcastHashJoin LeftOuter BuildRight (17)
                                 :                 :     :- ShuffleQueryStage (8), Statistics(sizeInBytes=704.0 B, rowCount=8)
                                 :                 :     :  +- Exchange (7)
                                 :                 :     :     +- * ColumnarToRow (6)
                                 :                 :     :        +- Scan parquet  (5)
                                 :                 :     +- BroadcastQueryStage (16), Statistics(sizeInBytes=2.1 MiB, rowCount=1.60E+3)
                                 :                 :        +- BroadcastExchange (15)
                                 :                 :           +- * Project (14)
                                 :                 :              +- ShuffleQueryStage (13), Statistics(sizeInBytes=215.5 KiB, rowCount=1.60E+3)
                                 :                 :                 +- Exchange (12)
                                 :                 :                    +- * Project (11)
                                 :                 :                       +- * ColumnarToRow (10)
                                 :                 :                          +- Scan parquet  (9)
                                 :                 +- BroadcastQueryStage (26), Statistics(sizeInBytes=2.0 MiB, rowCount=150)
                                 :                    +- BroadcastExchange (25)
                                 :                       +- ShuffleQueryStage (24), Statistics(sizeInBytes=89.7 KiB, rowCount=150)
                                 :                          +- Exchange (23)
                                 :                             +- * Project (22)
                                 :                                +- * Filter (21)
                                 :                                   +- * ColumnarToRow (20)
                                 :                                      +- Scan parquet  (19)
                                 +- BroadcastQueryStage (36), Statistics(sizeInBytes=2.0 MiB, rowCount=8)
                                    +- BroadcastExchange (35)
                                       +- * ColumnarToRow (34)
                                          +- Scan parquet  (33)
               +- == Initial Plan ==
                  HashAggregate (65)
                  +- HashAggregate (64)
                     +- Project (63)
                        +- Filter (62)
                           +- BroadcastHashJoin LeftOuter BuildRight (61)
                              :- Project (59)
                              :  +- Generate (58)
                              :     +- Project (57)
                              :        +- Generate (56)
                              :           +- Project (55)
                              :              +- BroadcastHashJoin LeftOuter BuildRight (54)
                              :                 :- Project (49)
                              :                 :  +- BroadcastHashJoin LeftOuter BuildRight (48)
                              :                 :     :- Exchange (43)
                              :                 :     :  +- Scan parquet  (5)
                              :                 :     +- BroadcastExchange (47)
                              :                 :        +- Project (46)
                              :                 :           +- Exchange (45)
                              :                 :              +- Project (44)
                              :                 :                 +- Scan parquet  (9)
                              :                 +- BroadcastExchange (53)
                              :                    +- Exchange (52)
                              :                       +- Project (51)
                              :                          +- Filter (50)
                              :                             +- Scan parquet  (19)
                              +- BroadcastExchange (60)
                                 +- Scan parquet  (33)


(1) EmptyRelation
Output: []
Arguments: [plan_id=21682909]

(2) ResultQueryStage
Output: []
Arguments: 1

(3) InMemoryTableScan
Output: []

(4) InMemoryRelation
Arguments: [product#15459015, plant#15459016], StorageLevel(disk, memory, 2 replicas)

(5) Scan parquet 
Output [2]: [product#15459015, plant#15459016]
Batched: true
Location: InMemoryFileIndex [hdlfs://332d7051-0d16-44f5-ae76-bf2dcf30365f.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000000246/shardId=0_1_10000000246]
ReadSchema: struct<product:string,plant:string>

(6) ColumnarToRow
Input [2]: [product#15459015, plant#15459016]

(7) Exchange
Input [2]: [product#15459015, plant#15459016]
Arguments: hashpartitioning(product#15459015, plant#15459016, 37), REPARTITION_BY_NUM, [plan_id=21682439]

(8) ShuffleQueryStage
Output [2]: [product#15459015, plant#15459016]
Arguments: 0

(9) Scan parquet 
Output [4]: [plant#15459022, product#15459023, productionAspect#15459025, systemId#15459020]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://332d7051-0d16-44f5-ae76-bf2dcf30365f.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>

(10) ColumnarToRow
Input [4]: [plant#15459022, product#15459023, productionAspect#15459025, systemId#15459020]

(11) Project
Output [4]: [plant#15459022.internalRefUUID AS _extract_internalRefUUID#15459516, product#15459023.internalRefUUID AS _extract_internalRefUUID#15459517, productionAspect#15459025.productMovementPlants.unitOfIssue.code AS _extract_code#15459518, productionAspect#15459025.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#15459519]
Input [4]: [plant#15459022, product#15459023, productionAspect#15459025, systemId#15459020]

(12) Exchange
Input [4]: [_extract_internalRefUUID#15459516, _extract_internalRefUUID#15459517, _extract_code#15459518, _extract_internalRefUUID#15459519]
Arguments: RoundRobinPartitioning(37), REPARTITION_BY_NUM, [plan_id=21682460]

(13) ShuffleQueryStage
Output [4]: [_extract_internalRefUUID#15459516, _extract_internalRefUUID#15459517, _extract_code#15459518, _extract_internalRefUUID#15459519]
Arguments: 1

(14) Project
Output [4]: [_extract_code#15459518, _extract_internalRefUUID#15459519, _extract_internalRefUUID#15459517, _extract_internalRefUUID#15459516]
Input [4]: [_extract_internalRefUUID#15459516, _extract_internalRefUUID#15459517, _extract_code#15459518, _extract_internalRefUUID#15459519]

(15) BroadcastExchange
Input [4]: [_extract_code#15459518, _extract_internalRefUUID#15459519, _extract_internalRefUUID#15459517, _extract_internalRefUUID#15459516]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=21682732]

(16) BroadcastQueryStage
Output [4]: [_extract_code#15459518, _extract_internalRefUUID#15459519, _extract_internalRefUUID#15459517, _extract_internalRefUUID#15459516]
Arguments: 5

(17) BroadcastHashJoin
Left keys [4]: [coalesce(product#15459015, ), isnull(product#15459015), coalesce(plant#15459016, ), isnull(plant#15459016)]
Right keys [4]: [coalesce(_extract_internalRefUUID#15459517, ), isnull(_extract_internalRefUUID#15459517), coalesce(_extract_internalRefUUID#15459516, ), isnull(_extract_internalRefUUID#15459516)]
Join type: LeftOuter
Join condition: None

(18) Project
Output [4]: [product#15459015, plant#15459016, _extract_code#15459518, _extract_internalRefUUID#15459519]
Input [6]: [product#15459015, plant#15459016, _extract_code#15459518, _extract_internalRefUUID#15459519, _extract_internalRefUUID#15459517, _extract_internalRefUUID#15459516]

(19) Scan parquet 
Output [4]: [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040, systemId#15459033]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://332d7051-0d16-44f5-ae76-bf2dcf30365f.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>

(20) ColumnarToRow
Input [4]: [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040, systemId#15459033]

(21) Filter
Input [4]: [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040, systemId#15459033]
Condition : isnotnull(internalUUID#15459034)

(22) Project
Output [3]: [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040]
Input [4]: [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040, systemId#15459033]

(23) Exchange
Input [3]: [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040]
Arguments: RoundRobinPartitioning(37), REPARTITION_BY_NUM, [plan_id=21682488]

(24) ShuffleQueryStage
Output [3]: [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040]
Arguments: 2

(25) BroadcastExchange
Input [3]: [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=21682498]

(26) BroadcastQueryStage
Output [3]: [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040]
Arguments: 4

(27) BroadcastHashJoin
Left keys [1]: [product#15459015]
Right keys [1]: [internalUUID#15459034]
Join type: LeftOuter
Join condition: None

(28) Project
Output [3]: [product#15459015, plant#15459016, filter(unitOfMeasureConversions#15459040, lambdafunction((((lambda u#15459047.measurementUnit1.code = _extract_code#15459518) AND (lambda u#15459047.measurementUnit1.internalRefUUID = _extract_internalRefUUID#15459519)) AND ((lambda u#15459047.measurementUnit2.code = baseUnitOfMeasure#15459038.code) AND (lambda u#15459047.measurementUnit2.internalRefUUID = baseUnitOfMeasure#15459038.internalRefUUID))), lambda u#15459047, false)) AS filteredUnitOfMeasureConversions#15459046]
Input [7]: [product#15459015, plant#15459016, _extract_code#15459518, _extract_internalRefUUID#15459519, internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040]

(29) Generate
Input [3]: [product#15459015, plant#15459016, filteredUnitOfMeasureConversions#15459046]
Arguments: explode(filteredUnitOfMeasureConversions#15459046.quantityNumerator), [product#15459015, plant#15459016, filteredUnitOfMeasureConversions#15459046], true, [quantityNumerator#15459054]

(30) Project
Output [3]: [product#15459015, plant#15459016, filteredUnitOfMeasureConversions#15459046.quantityDenominator AS _extract_quantityDenominator#15459437]
Input [4]: [product#15459015, plant#15459016, filteredUnitOfMeasureConversions#15459046, quantityNumerator#15459054]

(31) Generate
Input [3]: [product#15459015, plant#15459016, _extract_quantityDenominator#15459437]
Arguments: explode(_extract_quantityDenominator#15459437), [product#15459015, plant#15459016], true, [quantityDenominator#15459057]

(32) Project
Output [2]: [product#15459015, plant#15459016]
Input [3]: [product#15459015, plant#15459016, quantityDenominator#15459057]

(33) Scan parquet 
Output [4]: [product#15459402, plant#15459403, demandTimeBuckets#15459407, demandPointInTime#15459410]
Batched: true
Location: InMemoryFileIndex [hdlfs://332d7051-0d16-44f5-ae76-bf2dcf30365f.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-demand-service/out/demand/10000000246/0_1_10000000246]
ReadSchema: struct<product:string,plant:string,demandTimeBuckets:array<struct<demandStartDateTime:timestamp,demandQuantity:double,demandVariance:double>>,demandPointInTime:array<struct<demandDateTime:timestamp,demandQuantity:double,demandVariance:double>>>

(34) ColumnarToRow
Input [4]: [product#15459402, plant#15459403, demandTimeBuckets#15459407, demandPointInTime#15459410]

(35) BroadcastExchange
Input [4]: [product#15459402, plant#15459403, demandTimeBuckets#15459407, demandPointInTime#15459410]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[1, string, true], ), isnull(input[1, string, true]), coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=21682512]

(36) BroadcastQueryStage
Output [4]: [product#15459402, plant#15459403, demandTimeBuckets#15459407, demandPointInTime#15459410]
Arguments: 3

(37) BroadcastHashJoin
Left keys [4]: [coalesce(plant#15459016, ), isnull(plant#15459016), coalesce(product#15459015, ), isnull(product#15459015)]
Right keys [4]: [coalesce(plant#15459403, ), isnull(plant#15459403), coalesce(product#15459402, ), isnull(product#15459402)]
Join type: LeftOuter
Join condition: None

(38) Filter
Input [6]: [product#15459015, plant#15459016, product#15459402, plant#15459403, demandTimeBuckets#15459407, demandPointInTime#15459410]
Condition : (((size(demandTimeBuckets#15459407, true) = 0) OR isnull(demandTimeBuckets#15459407)) AND ((size(demandPointInTime#15459410, true) = 0) OR isnull(demandPointInTime#15459410)))

(39) Project
Output [2]: [product#15459015, plant#15459016]
Input [6]: [product#15459015, plant#15459016, product#15459402, plant#15459403, demandTimeBuckets#15459407, demandPointInTime#15459410]

(40) HashAggregate
Input [2]: [product#15459015, plant#15459016]
Keys [2]: [product#15459015, plant#15459016]
Functions: []
Aggregate Attributes: []
Results [2]: [product#15459015, plant#15459016]

(41) HashAggregate
Input [2]: [product#15459015, plant#15459016]
Keys [2]: [product#15459015, plant#15459016]
Functions: []
Aggregate Attributes: []
Results [2]: [product#15459015, plant#15459016]

(42) ResultQueryStage
Output [2]: [product#15459015, plant#15459016]
Arguments: 6

(43) Exchange
Input [2]: [product#15459015, plant#15459016]
Arguments: hashpartitioning(product#15459015, plant#15459016, 37), REPARTITION_BY_NUM, [plan_id=21682380]

(44) Project
Output [4]: [plant#15459022.internalRefUUID AS _extract_internalRefUUID#15459516, product#15459023.internalRefUUID AS _extract_internalRefUUID#15459517, productionAspect#15459025.productMovementPlants.unitOfIssue.code AS _extract_code#15459518, productionAspect#15459025.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#15459519]
Input [4]: [plant#15459022, product#15459023, productionAspect#15459025, systemId#15459020]

(45) Exchange
Input [4]: [_extract_internalRefUUID#15459516, _extract_internalRefUUID#15459517, _extract_code#15459518, _extract_internalRefUUID#15459519]
Arguments: RoundRobinPartitioning(37), REPARTITION_BY_NUM, [plan_id=21682383]

(46) Project
Output [4]: [_extract_code#15459518, _extract_internalRefUUID#15459519, _extract_internalRefUUID#15459517, _extract_internalRefUUID#15459516]
Input [4]: [_extract_internalRefUUID#15459516, _extract_internalRefUUID#15459517, _extract_code#15459518, _extract_internalRefUUID#15459519]

(47) BroadcastExchange
Input [4]: [_extract_code#15459518, _extract_internalRefUUID#15459519, _extract_internalRefUUID#15459517, _extract_internalRefUUID#15459516]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=21682404]

(48) BroadcastHashJoin
Left keys [4]: [coalesce(product#15459015, ), isnull(product#15459015), coalesce(plant#15459016, ), isnull(plant#15459016)]
Right keys [4]: [coalesce(_extract_internalRefUUID#15459517, ), isnull(_extract_internalRefUUID#15459517), coalesce(_extract_internalRefUUID#15459516, ), isnull(_extract_internalRefUUID#15459516)]
Join type: LeftOuter
Join condition: None

(49) Project
Output [4]: [product#15459015, plant#15459016, _extract_code#15459518, _extract_internalRefUUID#15459519]
Input [6]: [product#15459015, plant#15459016, _extract_code#15459518, _extract_internalRefUUID#15459519, _extract_internalRefUUID#15459517, _extract_internalRefUUID#15459516]

(50) Filter
Input [4]: [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040, systemId#15459033]
Condition : isnotnull(internalUUID#15459034)

(51) Project
Output [3]: [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040]
Input [4]: [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040, systemId#15459033]

(52) Exchange
Input [3]: [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040]
Arguments: RoundRobinPartitioning(37), REPARTITION_BY_NUM, [plan_id=21682390]

(53) BroadcastExchange
Input [3]: [internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=21682408]

(54) BroadcastHashJoin
Left keys [1]: [product#15459015]
Right keys [1]: [internalUUID#15459034]
Join type: LeftOuter
Join condition: None

(55) Project
Output [3]: [product#15459015, plant#15459016, filter(unitOfMeasureConversions#15459040, lambdafunction((((lambda u#15459047.measurementUnit1.code = _extract_code#15459518) AND (lambda u#15459047.measurementUnit1.internalRefUUID = _extract_internalRefUUID#15459519)) AND ((lambda u#15459047.measurementUnit2.code = baseUnitOfMeasure#15459038.code) AND (lambda u#15459047.measurementUnit2.internalRefUUID = baseUnitOfMeasure#15459038.internalRefUUID))), lambda u#15459047, false)) AS filteredUnitOfMeasureConversions#15459046]
Input [7]: [product#15459015, plant#15459016, _extract_code#15459518, _extract_internalRefUUID#15459519, internalUUID#15459034, baseUnitOfMeasure#15459038, unitOfMeasureConversions#15459040]

(56) Generate
Input [3]: [product#15459015, plant#15459016, filteredUnitOfMeasureConversions#15459046]
Arguments: explode(filteredUnitOfMeasureConversions#15459046.quantityNumerator), [product#15459015, plant#15459016, filteredUnitOfMeasureConversions#15459046], true, [quantityNumerator#15459054]

(57) Project
Output [3]: [product#15459015, plant#15459016, filteredUnitOfMeasureConversions#15459046.quantityDenominator AS _extract_quantityDenominator#15459437]
Input [4]: [product#15459015, plant#15459016, filteredUnitOfMeasureConversions#15459046, quantityNumerator#15459054]

(58) Generate
Input [3]: [product#15459015, plant#15459016, _extract_quantityDenominator#15459437]
Arguments: explode(_extract_quantityDenominator#15459437), [product#15459015, plant#15459016], true, [quantityDenominator#15459057]

(59) Project
Output [2]: [product#15459015, plant#15459016]
Input [3]: [product#15459015, plant#15459016, quantityDenominator#15459057]

(60) BroadcastExchange
Input [4]: [product#15459402, plant#15459403, demandTimeBuckets#15459407, demandPointInTime#15459410]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[1, string, true], ), isnull(input[1, string, true]), coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=21682416]

(61) BroadcastHashJoin
Left keys [4]: [coalesce(plant#15459016, ), isnull(plant#15459016), coalesce(product#15459015, ), isnull(product#15459015)]
Right keys [4]: [coalesce(plant#15459403, ), isnull(plant#15459403), coalesce(product#15459402, ), isnull(product#15459402)]
Join type: LeftOuter
Join condition: None

(62) Filter
Input [6]: [product#15459015, plant#15459016, product#15459402, plant#15459403, demandTimeBuckets#15459407, demandPointInTime#15459410]
Condition : (((size(demandTimeBuckets#15459407, true) = 0) OR isnull(demandTimeBuckets#15459407)) AND ((size(demandPointInTime#15459410, true) = 0) OR isnull(demandPointInTime#15459410)))

(63) Project
Output [2]: [product#15459015, plant#15459016]
Input [6]: [product#15459015, plant#15459016, product#15459402, plant#15459403, demandTimeBuckets#15459407, demandPointInTime#15459410]

(64) HashAggregate
Input [2]: [product#15459015, plant#15459016]
Keys [2]: [product#15459015, plant#15459016]
Functions: []
Aggregate Attributes: []
Results [2]: [product#15459015, plant#15459016]

(65) HashAggregate
Input [2]: [product#15459015, plant#15459016]
Keys [2]: [product#15459015, plant#15459016]
Functions: []
Aggregate Attributes: []
Results [2]: [product#15459015, plant#15459016]

(66) AdaptiveSparkPlan
Output [2]: [product#15459015, plant#15459016]
Arguments: isFinalPlan=true

(67) CollectLimit
Input: []
Arguments: 1

(68) AdaptiveSparkPlan
Output: []
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.activity.hdlfsEnabled	true
spark.activity.isolateSession	true
spark.sql.parquet.fieldId.read.enabled	true
spark.sql.parquet.fieldId.write.enabled	true
temporal.activity.id	46b22135-059c-30e8-ab0f-8081b745c2dd
temporal.activity.type	BufferDataSnapShot