root@6a301bd596e1:/# $SPARK_HOME/bin/spark-submit --master yarn --class com.example.geomesa.spark.CountByDay --jars /tmp/geomesa-hbase-spark-runtime-hbase2_2.12-3.3.0.jar --conf spark.serializer=org.apache.spark.serializer.KryoSerializer --conf spark.kryo.registrator=org.locationtech.geomesa.spark.GeoMesaSparkKryoRegistrator --driver-memory 4g --executor-memory 2g --executor-cores 1 --deploy-mode cluster /tmp/geomesa-examples-spark-3.3.0.jar 2021-10-07 15:11:34,943 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2021-10-07 15:11:36,227 INFO client.RMProxy: Connecting to ResourceManager at resourcemanager/192.168.16.4:8032 2021-10-07 15:11:36,620 INFO client.AHSProxy: Connecting to Application History server at historyserver/192.168.16.3:10200 2021-10-07 15:11:36,811 INFO yarn.Client: Requesting a new application from cluster with 1 NodeManagers 2021-10-07 15:11:37,732 INFO conf.Configuration: resource-types.xml not found 2021-10-07 15:11:37,732 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'. 2021-10-07 15:11:37,777 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (8192 MB per container) 2021-10-07 15:11:37,778 INFO yarn.Client: Will allocate AM container, with 4505 MB memory including 409 MB overhead 2021-10-07 15:11:37,779 INFO yarn.Client: Setting up container launch context for our AM 2021-10-07 15:11:37,781 INFO yarn.Client: Setting up the launch environment for our AM container 2021-10-07 15:11:37,796 INFO yarn.Client: Preparing resources for our AM container 2021-10-07 15:11:37,890 WARN yarn.Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME. 2021-10-07 15:11:48,266 INFO yarn.Client: Uploading resource file:/tmp/spark-83839800-2f25-4940-8ea1-fa182547effd/__spark_libs__4086547486002414252.zip -> hdfs://namenode:9000/user/root/.sparkStaging/application_1633616399531_0007/__spark_libs__4086547486002414252.zip 2021-10-07 15:11:51,177 INFO yarn.Client: Uploading resource file:/tmp/geomesa-examples-spark-3.3.0.jar -> hdfs://namenode:9000/user/root/.sparkStaging/application_1633616399531_0007/geomesa-examples-spark-3.3.0.jar 2021-10-07 15:11:51,689 INFO yarn.Client: Uploading resource file:/tmp/geomesa-hbase-spark-runtime-hbase2_2.12-3.3.0.jar -> hdfs://namenode:9000/user/root/.sparkStaging/application_1633616399531_0007/geomesa-hbase-spark-runtime-hbase2_2.12-3.3.0.jar 2021-10-07 15:11:52,852 INFO yarn.Client: Uploading resource file:/tmp/spark-83839800-2f25-4940-8ea1-fa182547effd/__spark_conf__7211685152103377471.zip -> hdfs://namenode:9000/user/root/.sparkStaging/application_1633616399531_0007/__spark_conf__.zip 2021-10-07 15:11:52,944 INFO spark.SecurityManager: Changing view acls to: root 2021-10-07 15:11:52,945 INFO spark.SecurityManager: Changing modify acls to: root 2021-10-07 15:11:52,947 INFO spark.SecurityManager: Changing view acls groups to: 2021-10-07 15:11:52,948 INFO spark.SecurityManager: Changing modify acls groups to: 2021-10-07 15:11:52,949 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set() 2021-10-07 15:11:53,054 INFO yarn.Client: Submitting application application_1633616399531_0007 to ResourceManager 2021-10-07 15:11:53,360 INFO impl.YarnClientImpl: Submitted application application_1633616399531_0007 2021-10-07 15:11:54,369 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:11:54,376 INFO yarn.Client: client token: N/A diagnostics: AM container is launched, waiting for AM container to Register with RM ApplicationMaster host: N/A ApplicationMaster RPC port: -1 queue: default start time: 1633619513090 final status: UNDEFINED tracking URL: http://resourcemanager:8088/proxy/application_1633616399531_0007/ user: root 2021-10-07 15:11:55,382 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:11:56,388 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:11:57,393 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:11:58,399 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:11:59,411 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:00,422 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:01,428 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:02,443 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:03,450 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:04,459 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:05,483 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:06,490 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:07,505 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:08,523 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:09,578 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:10,592 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:11,602 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:12,608 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:13,618 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:14,627 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:15,634 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:12:16,640 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:16,642 INFO yarn.Client: client token: N/A diagnostics: N/A ApplicationMaster host: f9116b789939 ApplicationMaster RPC port: 42771 queue: default start time: 1633619513090 final status: UNDEFINED tracking URL: http://resourcemanager:8088/proxy/application_1633616399531_0007/ user: root 2021-10-07 15:12:17,648 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:18,652 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:19,909 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:20,926 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:21,901 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:22,906 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:23,911 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:24,920 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:25,975 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:27,004 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:28,008 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:29,012 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:30,015 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:31,035 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:32,079 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:33,087 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:34,113 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:35,130 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:36,135 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:37,141 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:38,145 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:39,163 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:40,176 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:41,184 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:42,192 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:43,204 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:44,211 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:45,216 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:46,222 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:47,225 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:48,232 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:49,241 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:50,249 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:51,258 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:52,234 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:53,245 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:54,253 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:55,261 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:56,269 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:57,276 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:58,286 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:12:59,295 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:00,302 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:01,308 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:02,317 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:03,321 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:04,332 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:05,341 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:06,354 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:07,364 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:08,378 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:09,385 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:10,393 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:11,405 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:12,413 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:13,421 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:14,433 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:15,440 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:16,449 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:17,457 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:18,465 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:19,473 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:20,482 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:21,447 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:22,453 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:23,463 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:24,469 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:25,478 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:26,487 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:27,491 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:28,498 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:29,510 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:30,517 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:31,527 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:32,533 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:33,541 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:34,547 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:35,552 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:36,560 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:37,567 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:38,575 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:39,583 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:40,591 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:41,602 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:42,614 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:43,622 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:44,631 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:45,640 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:46,647 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:47,654 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:48,661 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:49,671 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:50,680 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:51,645 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:52,651 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:53,656 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:54,662 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:55,669 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:13:56,676 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:13:56,682 INFO yarn.Client: client token: N/A diagnostics: [Thu Oct 07 15:13:56 +0000 2021] Application is Activated, waiting for resources to be assigned for AM. Details : AM Partition = ; Partition Resource = ; Queue's Absolute capacity = 100.0 % ; Queue's Absolute used capacity = 0.0 % ; Queue's Absolute max capacity = 100.0 % ; Queue's capacity (absolute resource) = ; Queue's used capacity (absolute resource) = ; Queue's max capacity (absolute resource) = ; ApplicationMaster host: N/A ApplicationMaster RPC port: -1 queue: default start time: 1633619513090 final status: UNDEFINED tracking URL: http://resourcemanager:8088/proxy/application_1633616399531_0007/ user: root 2021-10-07 15:13:57,688 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:13:58,691 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:13:59,696 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:14:00,703 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:14:01,710 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:14:02,716 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:14:03,722 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:14:04,727 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:14:05,732 INFO yarn.Client: Application report for application_1633616399531_0007 (state: ACCEPTED) 2021-10-07 15:14:06,736 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:06,737 INFO yarn.Client: client token: N/A diagnostics: N/A ApplicationMaster host: f9116b789939 ApplicationMaster RPC port: 33409 queue: default start time: 1633619513090 final status: UNDEFINED tracking URL: http://resourcemanager:8088/proxy/application_1633616399531_0007/ user: root 2021-10-07 15:14:07,744 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:08,749 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:09,753 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:10,756 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:11,762 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:12,765 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:13,770 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:14,775 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:15,780 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:16,784 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:17,790 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:18,796 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:19,803 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:20,813 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:21,789 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:22,797 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:23,802 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:24,808 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:25,814 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:26,822 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:27,826 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:28,834 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:29,841 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:30,860 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:31,868 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:32,884 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:33,897 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:34,910 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:35,925 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:36,933 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:37,941 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:38,949 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:39,957 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:40,973 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:41,979 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:42,991 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:43,999 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:45,009 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:46,015 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:47,022 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:48,037 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:49,050 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:50,058 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:51,068 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:52,040 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:53,052 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:54,064 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:55,073 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:56,079 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:57,085 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:58,090 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:14:59,098 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:00,102 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:01,116 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:02,123 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:03,127 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:04,134 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:05,149 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:06,155 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:07,159 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:08,170 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:09,176 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:10,183 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:11,192 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:12,198 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:13,207 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:14,215 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:15,220 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:16,225 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:17,231 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:18,238 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:19,246 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:20,253 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:21,259 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:22,234 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:23,241 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:24,248 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:25,255 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:26,264 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:27,270 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:28,276 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:29,279 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:30,286 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:31,295 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:32,301 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:33,306 INFO yarn.Client: Application report for application_1633616399531_0007 (state: RUNNING) 2021-10-07 15:15:34,312 INFO yarn.Client: Application report for application_1633616399531_0007 (state: FINISHED) 2021-10-07 15:15:34,314 INFO yarn.Client: client token: N/A diagnostics: User class threw exception: org.apache.hadoop.hbase.client.RetriesExhaustedException: Cannot get the location for replica0 of region for zpw_gdelt_2dquickstart_z3_geom_dtg_v7,, in hbase:meta at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:336) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:162) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:58) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithoutRetries(RpcRetryingCallerImpl.java:192) at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:267) at org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:435) at org.apache.hadoop.hbase.client.ClientScanner.nextWithSyncCache(ClientScanner.java:310) at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:595) at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:822) at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:790) at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:743) at org.apache.hadoop.hbase.MetaTableAccessor.scanMetaForTableRegions(MetaTableAccessor.java:738) at org.apache.hadoop.hbase.client.HRegionLocator.listRegionLocations(HRegionLocator.java:114) at org.apache.hadoop.hbase.client.HRegionLocator.getAllRegionLocations(HRegionLocator.java:78) at org.apache.hadoop.hbase.mapreduce.RegionSizeCalculator.getRegionServersOfTable(RegionSizeCalculator.java:103) at org.apache.hadoop.hbase.mapreduce.RegionSizeCalculator.init(RegionSizeCalculator.java:79) at org.apache.hadoop.hbase.mapreduce.RegionSizeCalculator.(RegionSizeCalculator.java:61) at org.apache.hadoop.hbase.mapreduce.MultiTableInputFormatBase.getSplits(MultiTableInputFormatBase.java:190) at org.locationtech.geomesa.hbase.jobs.GeoMesaHBaseInputFormat.getSplits(GeoMesaHBaseInputFormat.scala:43) at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:131) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.rdd.RDD.partitions(RDD.scala:296) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.rdd.RDD.partitions(RDD.scala:296) at org.locationtech.geomesa.spark.SpatialRDD.getPartitions(SpatialRDD.scala:39) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.rdd.RDD.partitions(RDD.scala:296) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.rdd.RDD.partitions(RDD.scala:296) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.rdd.RDD.partitions(RDD.scala:296) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.rdd.RDD.partitions(RDD.scala:296) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.rdd.RDD.partitions(RDD.scala:296) at org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:442) at org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:425) at org.apache.spark.sql.execution.CollectLimitExec.executeCollect(limit.scala:47) at org.apache.spark.sql.Dataset.collectFromPlan(Dataset.scala:3696) at org.apache.spark.sql.Dataset.$anonfun$head$1(Dataset.scala:2722) at org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:3687) at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:103) at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:163) at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:90) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775) at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64) at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3685) at org.apache.spark.sql.Dataset.head(Dataset.scala:2722) at org.apache.spark.sql.Dataset.take(Dataset.scala:2929) at org.apache.spark.sql.Dataset.getRows(Dataset.scala:301) at org.apache.spark.sql.Dataset.showString(Dataset.scala:338) at org.apache.spark.sql.Dataset.show(Dataset.scala:825) at org.apache.spark.sql.Dataset.show(Dataset.scala:784) at com.example.geomesa.spark.CountByDay$.main(CountByDay.scala:58) at com.example.geomesa.spark.CountByDay.main(CountByDay.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:732) Caused by: java.io.IOException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase at org.apache.hadoop.hbase.client.ConnectionImplementation.get(ConnectionImplementation.java:2130) at org.apache.hadoop.hbase.client.ConnectionImplementation.locateMeta(ConnectionImplementation.java:823) at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:790) at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:328) ... 71 more Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase at org.apache.zookeeper.KeeperException.create(KeeperException.java:102) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$ZKTask$1.exec(ReadOnlyZKClient.java:198) at org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:342) at java.lang.Thread.run(Thread.java:748) ApplicationMaster host: f9116b789939 ApplicationMaster RPC port: 33409 queue: default start time: 1633619513090 final status: FAILED tracking URL: http://resourcemanager:8088/proxy/application_1633616399531_0007/ user: root 2021-10-07 15:15:34,401 ERROR yarn.Client: Application diagnostics message: User class threw exception: org.apache.hadoop.hbase.client.RetriesExhaustedException: Cannot get the location for replica0 of region for zpw_gdelt_2dquickstart_z3_geom_dtg_v7,, in hbase:meta at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:336) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:162) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:58) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithoutRetries(RpcRetryingCallerImpl.java:192) at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:267) at org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:435) at org.apache.hadoop.hbase.client.ClientScanner.nextWithSyncCache(ClientScanner.java:310) at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:595) at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:822) at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:790) at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:743) at org.apache.hadoop.hbase.MetaTableAccessor.scanMetaForTableRegions(MetaTableAccessor.java:738) at org.apache.hadoop.hbase.client.HRegionLocator.listRegionLocations(HRegionLocator.java:114) at org.apache.hadoop.hbase.client.HRegionLocator.getAllRegionLocations(HRegionLocator.java:78) at org.apache.hadoop.hbase.mapreduce.RegionSizeCalculator.getRegionServersOfTable(RegionSizeCalculator.java:103) at org.apache.hadoop.hbase.mapreduce.RegionSizeCalculator.init(RegionSizeCalculator.java:79) at org.apache.hadoop.hbase.mapreduce.RegionSizeCalculator.(RegionSizeCalculator.java:61) at org.apache.hadoop.hbase.mapreduce.MultiTableInputFormatBase.getSplits(MultiTableInputFormatBase.java:190) at org.locationtech.geomesa.hbase.jobs.GeoMesaHBaseInputFormat.getSplits(GeoMesaHBaseInputFormat.scala:43) at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:131) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.rdd.RDD.partitions(RDD.scala:296) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.rdd.RDD.partitions(RDD.scala:296) at org.locationtech.geomesa.spark.SpatialRDD.getPartitions(SpatialRDD.scala:39) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.rdd.RDD.partitions(RDD.scala:296) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.rdd.RDD.partitions(RDD.scala:296) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.rdd.RDD.partitions(RDD.scala:296) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.rdd.RDD.partitions(RDD.scala:296) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.rdd.RDD.partitions(RDD.scala:296) at org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:442) at org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:425) at org.apache.spark.sql.execution.CollectLimitExec.executeCollect(limit.scala:47) at org.apache.spark.sql.Dataset.collectFromPlan(Dataset.scala:3696) at org.apache.spark.sql.Dataset.$anonfun$head$1(Dataset.scala:2722) at org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:3687) at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:103) at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:163) at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:90) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775) at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64) at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3685) at org.apache.spark.sql.Dataset.head(Dataset.scala:2722) at org.apache.spark.sql.Dataset.take(Dataset.scala:2929) at org.apache.spark.sql.Dataset.getRows(Dataset.scala:301) at org.apache.spark.sql.Dataset.showString(Dataset.scala:338) at org.apache.spark.sql.Dataset.show(Dataset.scala:825) at org.apache.spark.sql.Dataset.show(Dataset.scala:784) at com.example.geomesa.spark.CountByDay$.main(CountByDay.scala:58) at com.example.geomesa.spark.CountByDay.main(CountByDay.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:732) Caused by: java.io.IOException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase at org.apache.hadoop.hbase.client.ConnectionImplementation.get(ConnectionImplementation.java:2130) at org.apache.hadoop.hbase.client.ConnectionImplementation.locateMeta(ConnectionImplementation.java:823) at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:790) at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:328) ... 71 more Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase at org.apache.zookeeper.KeeperException.create(KeeperException.java:102) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$ZKTask$1.exec(ReadOnlyZKClient.java:198) at org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:342) at java.lang.Thread.run(Thread.java:748) Exception in thread "main" org.apache.spark.SparkException: Application application_1633616399531_0007 finished with failed status at org.apache.spark.deploy.yarn.Client.run(Client.scala:1242) at org.apache.spark.deploy.yarn.YarnClusterApplication.start(Client.scala:1634) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1039) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) 2021-10-07 15:15:34,447 INFO util.ShutdownHookManager: Shutdown hook called 2021-10-07 15:15:34,451 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-83839800-2f25-4940-8ea1-fa182547effd 2021-10-07 15:15:34,469 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-adffb12b-54a2-4371-b609-d0cf658722fd root@6a301bd596e1:/#