3.5.5
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
crp-wrkload-determination
application UI
Details for Stage 118950 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
18 ms
Locality Level Summary:
Process local: 200
Input Size / Records:
3.1 KiB / 0
Shuffle Write Size / Records:
14.1 KiB / 200
Associated Job Ids:
46389
DAG Visualization
digraph G { subgraph clusterstage_118950 { label="Stage 118950"; subgraph cluster559528 { label="Exchange"; 201068 [labelType="html" label="ShuffledRowRDD [201068] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster559524 { label="WholeStageCodegen (1)"; 201069 [labelType="html" label="MapPartitionsRDD [201069] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster559523 { label="MapGroups"; 201070 [labelType="html" label="MapPartitionsRDD [201070] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster559520 { label="WholeStageCodegen (2)"; 201071 [labelType="html" label="MapPartitionsRDD [201071] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster559529 { label="mapPartitionsInternal"; 201072 [labelType="html" label="MapPartitionsRDD [201072] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster559530 { label="mapPartitionsInternal"; 201073 [labelType="html" label="AdaptiveSparkPlan isFinalPlan=false\n+- SerializeFromObject [staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, knownnotnull(assertnotnull(input[0, org.apache.spark.sql.delta.SerializableFileStatus, true])).path, true, false, true) AS path#3266915, knownnotnull(assertnotnull(input[0, org.apache.spark.sql.delta.SerializableFileStatus, true])).length AS length#3266916L, knownnotnull(assertnotnull(input[0, org.apache.spark.sql.delta.SerializableFileStatus, true])).isDir AS isDir#3266917, knownnotnull(assertnotnull(input[0, org.apache.spark.sql.delta.SerializableFileStatus, true])).modificationTime AS modificationTime#3266918L]\n +- MapGroups org.apache.spark.sql.KeyValueGroupedDataset$$Lambda$6934/0x0000000801d21280@7fa4d7ef, value#3266909.toString, newInstance(class org.apache.spark.sql.delta.SerializableFileStatus), [value#3266909], [path#3266899, length#3266900L, isDir#3266901, modificationTime#3266902L], obj#3266914: org.apache.spark.sql.delta.SerializableFileStatus\n ... [201073] [Cached] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster559538 { label="InMemoryTableScan"; 201074 [labelType="html" label="MapPartitionsRDD [201074] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; 201075 [labelType="html" label="MapPartitionsRDD [201075] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; 201076 [labelType="html" label="MapPartitionsRDD [201076] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster559532 { label="WholeStageCodegen (1)"; 201077 [labelType="html" label="MapPartitionsRDD [201077] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster559531 { label="Exchange"; 201078 [labelType="html" label="MapPartitionsRDD [201078] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } } 201068->201069; 201069->201070; 201070->201071; 201071->201072; 201072->201073; 201073->201074; 201074->201075; 201075->201076; 201076->201077; 201077->201078; }
201067,201068
201073
Event Timeline
Enable zooming
Tasks: 200. 2 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time