2023-02-10 08:00:07,302 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4 2023-02-10 08:00:07,316 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2 timeout: 13 mins 2023-02-10 08:00:07,355 INFO [Time-limited test] hbase.ResourceChecker(147): before: client.TestAsyncClusterAdminApi2#testStop Thread=8, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=239, ProcessCount=171, AvailableMemoryMB=4018 2023-02-10 08:00:07,362 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-10 08:00:07,362 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/cluster_489b6fd9-e39e-448e-c70d-be315a99e8c1, deleteOnExit=true 2023-02-10 08:00:07,362 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-10 08:00:07,363 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/test.cache.data in system properties and HBase conf 2023-02-10 08:00:07,363 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/hadoop.tmp.dir in system properties and HBase conf 2023-02-10 08:00:07,364 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/hadoop.log.dir in system properties and HBase conf 2023-02-10 08:00:07,364 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-10 08:00:07,365 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-10 08:00:07,365 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-10 08:00:07,488 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-10 08:00:07,870 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-10 08:00:07,874 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-10 08:00:07,874 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-10 08:00:07,874 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-10 08:00:07,875 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-10 08:00:07,875 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-10 08:00:07,875 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-10 08:00:07,875 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-10 08:00:07,876 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-10 08:00:07,876 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-10 08:00:07,876 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/nfs.dump.dir in system properties and HBase conf 2023-02-10 08:00:07,877 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/java.io.tmpdir in system properties and HBase conf 2023-02-10 08:00:07,877 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-10 08:00:07,877 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-10 08:00:07,877 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-10 08:00:08,338 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-10 08:00:08,343 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-10 08:00:08,628 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-10 08:00:08,776 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-10 08:00:08,790 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-10 08:00:08,827 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-10 08:00:08,859 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/java.io.tmpdir/Jetty_localhost_42069_hdfs____.qm00bl/webapp 2023-02-10 08:00:09,013 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42069 2023-02-10 08:00:09,023 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-10 08:00:09,024 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-10 08:00:09,458 WARN [Listener at localhost/44445] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-10 08:00:09,524 WARN [Listener at localhost/44445] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-10 08:00:09,543 WARN [Listener at localhost/44445] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-10 08:00:09,551 INFO [Listener at localhost/44445] log.Slf4jLog(67): jetty-6.1.26 2023-02-10 08:00:09,555 INFO [Listener at localhost/44445] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/java.io.tmpdir/Jetty_localhost_44463_datanode____eddyhk/webapp 2023-02-10 08:00:09,651 INFO [Listener at localhost/44445] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44463 2023-02-10 08:00:09,957 WARN [Listener at localhost/39929] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-10 08:00:09,969 WARN [Listener at localhost/39929] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-10 08:00:09,976 WARN [Listener at localhost/39929] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-10 08:00:09,978 INFO [Listener at localhost/39929] log.Slf4jLog(67): jetty-6.1.26 2023-02-10 08:00:09,985 INFO [Listener at localhost/39929] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/java.io.tmpdir/Jetty_localhost_46489_datanode____mohcre/webapp 2023-02-10 08:00:10,109 INFO [Listener at localhost/39929] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46489 2023-02-10 08:00:10,117 WARN [Listener at localhost/38269] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-10 08:00:10,134 WARN [Listener at localhost/38269] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-10 08:00:10,138 WARN [Listener at localhost/38269] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-10 08:00:10,141 INFO [Listener at localhost/38269] log.Slf4jLog(67): jetty-6.1.26 2023-02-10 08:00:10,149 INFO [Listener at localhost/38269] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/java.io.tmpdir/Jetty_localhost_37389_datanode____.x2rcsp/webapp 2023-02-10 08:00:10,283 INFO [Listener at localhost/38269] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37389 2023-02-10 08:00:10,295 WARN [Listener at localhost/33663] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-10 08:00:10,439 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x809b058f703c0730: Processing first storage report for DS-e3349651-65d4-4752-b525-e3f72b4f516f from datanode 25d7c9d1-863e-4964-bc32-77dcf2a8a658 2023-02-10 08:00:10,440 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x809b058f703c0730: from storage DS-e3349651-65d4-4752-b525-e3f72b4f516f node DatanodeRegistration(127.0.0.1:43743, datanodeUuid=25d7c9d1-863e-4964-bc32-77dcf2a8a658, infoPort=36175, infoSecurePort=0, ipcPort=33663, storageInfo=lv=-57;cid=testClusterID;nsid=1840699377;c=1676016008415), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-10 08:00:10,440 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x2fdb07ed50584550: Processing first storage report for DS-280ee229-54a0-44df-8128-73e286576e3a from datanode e08ec0d9-4f28-4dab-8a37-7cbe6cb2d5f1 2023-02-10 08:00:10,441 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x2fdb07ed50584550: from storage DS-280ee229-54a0-44df-8128-73e286576e3a node DatanodeRegistration(127.0.0.1:40649, datanodeUuid=e08ec0d9-4f28-4dab-8a37-7cbe6cb2d5f1, infoPort=34165, infoSecurePort=0, ipcPort=38269, storageInfo=lv=-57;cid=testClusterID;nsid=1840699377;c=1676016008415), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 08:00:10,441 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x89098aa4e5c13430: Processing first storage report for DS-dce3e251-5265-4dc8-924f-b4ded3491509 from datanode d25a848a-2cc2-4067-ad81-2ac90a87c200 2023-02-10 08:00:10,441 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x89098aa4e5c13430: from storage DS-dce3e251-5265-4dc8-924f-b4ded3491509 node DatanodeRegistration(127.0.0.1:40317, datanodeUuid=d25a848a-2cc2-4067-ad81-2ac90a87c200, infoPort=42399, infoSecurePort=0, ipcPort=39929, storageInfo=lv=-57;cid=testClusterID;nsid=1840699377;c=1676016008415), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 08:00:10,441 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x809b058f703c0730: Processing first storage report for DS-94a29101-f162-466c-be25-c12cbb90e7d2 from datanode 25d7c9d1-863e-4964-bc32-77dcf2a8a658 2023-02-10 08:00:10,441 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x809b058f703c0730: from storage DS-94a29101-f162-466c-be25-c12cbb90e7d2 node DatanodeRegistration(127.0.0.1:43743, datanodeUuid=25d7c9d1-863e-4964-bc32-77dcf2a8a658, infoPort=36175, infoSecurePort=0, ipcPort=33663, storageInfo=lv=-57;cid=testClusterID;nsid=1840699377;c=1676016008415), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 08:00:10,441 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x2fdb07ed50584550: Processing first storage report for DS-dfb505be-139c-4286-adbb-81b2c1a58751 from datanode e08ec0d9-4f28-4dab-8a37-7cbe6cb2d5f1 2023-02-10 08:00:10,441 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x2fdb07ed50584550: from storage DS-dfb505be-139c-4286-adbb-81b2c1a58751 node DatanodeRegistration(127.0.0.1:40649, datanodeUuid=e08ec0d9-4f28-4dab-8a37-7cbe6cb2d5f1, infoPort=34165, infoSecurePort=0, ipcPort=38269, storageInfo=lv=-57;cid=testClusterID;nsid=1840699377;c=1676016008415), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 08:00:10,441 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x89098aa4e5c13430: Processing first storage report for DS-3910a9cf-f1b6-46df-97d0-c7efebb2a3ba from datanode d25a848a-2cc2-4067-ad81-2ac90a87c200 2023-02-10 08:00:10,441 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x89098aa4e5c13430: from storage DS-3910a9cf-f1b6-46df-97d0-c7efebb2a3ba node DatanodeRegistration(127.0.0.1:40317, datanodeUuid=d25a848a-2cc2-4067-ad81-2ac90a87c200, infoPort=42399, infoSecurePort=0, ipcPort=39929, storageInfo=lv=-57;cid=testClusterID;nsid=1840699377;c=1676016008415), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 08:00:10,682 DEBUG [Listener at localhost/33663] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4 2023-02-10 08:00:10,744 INFO [Listener at localhost/33663] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/cluster_489b6fd9-e39e-448e-c70d-be315a99e8c1/zookeeper_0, clientPort=60578, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/cluster_489b6fd9-e39e-448e-c70d-be315a99e8c1/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/cluster_489b6fd9-e39e-448e-c70d-be315a99e8c1/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-10 08:00:10,758 INFO [Listener at localhost/33663] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=60578 2023-02-10 08:00:10,765 INFO [Listener at localhost/33663] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:10,767 INFO [Listener at localhost/33663] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:11,448 INFO [Listener at localhost/33663] util.FSUtils(479): Created version file at hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12 with version=8 2023-02-10 08:00:11,448 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/hbase-staging 2023-02-10 08:00:11,750 INFO [Listener at localhost/33663] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-10 08:00:12,209 INFO [Listener at localhost/33663] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=6 2023-02-10 08:00:12,241 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:12,242 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:12,242 INFO [Listener at localhost/33663] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-10 08:00:12,242 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:12,242 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-10 08:00:12,384 INFO [Listener at localhost/33663] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-10 08:00:12,459 DEBUG [Listener at localhost/33663] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-10 08:00:12,555 INFO [Listener at localhost/33663] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45287 2023-02-10 08:00:12,565 INFO [Listener at localhost/33663] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:12,567 INFO [Listener at localhost/33663] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:12,592 INFO [Listener at localhost/33663] zookeeper.RecoverableZooKeeper(93): Process identifier=master:45287 connecting to ZooKeeper ensemble=127.0.0.1:60578 2023-02-10 08:00:12,635 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:452870x0, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-10 08:00:12,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:45287-0x1003d3eafaf0000 connected 2023-02-10 08:00:12,662 DEBUG [Listener at localhost/33663] zookeeper.ZKUtil(164): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 08:00:12,663 DEBUG [Listener at localhost/33663] zookeeper.ZKUtil(164): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 08:00:12,666 DEBUG [Listener at localhost/33663] zookeeper.ZKUtil(164): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-10 08:00:12,674 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45287 2023-02-10 08:00:12,674 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45287 2023-02-10 08:00:12,675 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45287 2023-02-10 08:00:12,675 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45287 2023-02-10 08:00:12,675 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45287 2023-02-10 08:00:12,681 INFO [Listener at localhost/33663] master.HMaster(439): hbase.rootdir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12, hbase.cluster.distributed=false 2023-02-10 08:00:12,750 INFO [Listener at localhost/33663] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-10 08:00:12,750 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:12,751 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:12,751 INFO [Listener at localhost/33663] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-10 08:00:12,751 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:12,751 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-10 08:00:12,756 INFO [Listener at localhost/33663] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-10 08:00:12,759 INFO [Listener at localhost/33663] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36831 2023-02-10 08:00:12,762 INFO [Listener at localhost/33663] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-10 08:00:12,768 DEBUG [Listener at localhost/33663] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-10 08:00:12,770 INFO [Listener at localhost/33663] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:12,771 INFO [Listener at localhost/33663] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:12,774 INFO [Listener at localhost/33663] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36831 connecting to ZooKeeper ensemble=127.0.0.1:60578 2023-02-10 08:00:12,778 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:368310x0, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-10 08:00:12,779 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:36831-0x1003d3eafaf0001 connected 2023-02-10 08:00:12,779 DEBUG [Listener at localhost/33663] zookeeper.ZKUtil(164): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 08:00:12,780 DEBUG [Listener at localhost/33663] zookeeper.ZKUtil(164): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 08:00:12,781 DEBUG [Listener at localhost/33663] zookeeper.ZKUtil(164): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-10 08:00:12,782 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36831 2023-02-10 08:00:12,782 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36831 2023-02-10 08:00:12,782 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36831 2023-02-10 08:00:12,784 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36831 2023-02-10 08:00:12,784 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36831 2023-02-10 08:00:12,798 INFO [Listener at localhost/33663] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-10 08:00:12,798 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:12,799 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:12,799 INFO [Listener at localhost/33663] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-10 08:00:12,800 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:12,800 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-10 08:00:12,800 INFO [Listener at localhost/33663] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-10 08:00:12,802 INFO [Listener at localhost/33663] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34749 2023-02-10 08:00:12,802 INFO [Listener at localhost/33663] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-10 08:00:12,803 DEBUG [Listener at localhost/33663] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-10 08:00:12,804 INFO [Listener at localhost/33663] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:12,807 INFO [Listener at localhost/33663] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:12,810 INFO [Listener at localhost/33663] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34749 connecting to ZooKeeper ensemble=127.0.0.1:60578 2023-02-10 08:00:12,813 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:347490x0, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-10 08:00:12,814 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:34749-0x1003d3eafaf0002 connected 2023-02-10 08:00:12,814 DEBUG [Listener at localhost/33663] zookeeper.ZKUtil(164): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 08:00:12,815 DEBUG [Listener at localhost/33663] zookeeper.ZKUtil(164): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 08:00:12,816 DEBUG [Listener at localhost/33663] zookeeper.ZKUtil(164): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-10 08:00:12,817 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34749 2023-02-10 08:00:12,817 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34749 2023-02-10 08:00:12,817 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34749 2023-02-10 08:00:12,818 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34749 2023-02-10 08:00:12,818 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34749 2023-02-10 08:00:12,832 INFO [Listener at localhost/33663] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-10 08:00:12,832 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:12,832 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:12,832 INFO [Listener at localhost/33663] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-10 08:00:12,832 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:12,832 INFO [Listener at localhost/33663] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-10 08:00:12,833 INFO [Listener at localhost/33663] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-10 08:00:12,834 INFO [Listener at localhost/33663] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35917 2023-02-10 08:00:12,834 INFO [Listener at localhost/33663] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-10 08:00:12,836 DEBUG [Listener at localhost/33663] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-10 08:00:12,837 INFO [Listener at localhost/33663] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:12,838 INFO [Listener at localhost/33663] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:12,840 INFO [Listener at localhost/33663] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35917 connecting to ZooKeeper ensemble=127.0.0.1:60578 2023-02-10 08:00:12,843 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:359170x0, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-10 08:00:12,844 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:35917-0x1003d3eafaf0003 connected 2023-02-10 08:00:12,845 DEBUG [Listener at localhost/33663] zookeeper.ZKUtil(164): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 08:00:12,845 DEBUG [Listener at localhost/33663] zookeeper.ZKUtil(164): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 08:00:12,846 DEBUG [Listener at localhost/33663] zookeeper.ZKUtil(164): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-10 08:00:12,847 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35917 2023-02-10 08:00:12,847 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35917 2023-02-10 08:00:12,848 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35917 2023-02-10 08:00:12,848 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35917 2023-02-10 08:00:12,848 DEBUG [Listener at localhost/33663] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35917 2023-02-10 08:00:12,850 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,45287,1676016011592 2023-02-10 08:00:12,860 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-10 08:00:12,861 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,45287,1676016011592 2023-02-10 08:00:12,879 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-10 08:00:12,879 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-10 08:00:12,879 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-10 08:00:12,879 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-10 08:00:12,880 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:12,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-10 08:00:12,882 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,45287,1676016011592 from backup master directory 2023-02-10 08:00:12,882 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-10 08:00:12,886 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,45287,1676016011592 2023-02-10 08:00:12,886 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-10 08:00:12,887 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-10 08:00:12,887 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,45287,1676016011592 2023-02-10 08:00:12,890 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-10 08:00:12,892 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-10 08:00:12,983 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/hbase.id with ID: 855fac21-b430-4aac-843a-17f72c85fd80 2023-02-10 08:00:13,026 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:13,042 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:13,089 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2e677d8b to 127.0.0.1:60578 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 08:00:13,123 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@21c54776, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 08:00:13,147 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-10 08:00:13,148 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-10 08:00:13,166 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-10 08:00:13,166 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-10 08:00:13,168 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-10 08:00:13,172 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-10 08:00:13,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 08:00:13,207 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/MasterData/data/master/store-tmp 2023-02-10 08:00:13,243 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 08:00:13,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-10 08:00:13,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:13,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:13,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-10 08:00:13,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:13,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:13,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676016013243Disabling compacts and flushes for region at 1676016013244 (+1 ms)Disabling writes for close at 1676016013244Writing region close event to WAL at 1676016013244Closed at 1676016013244 2023-02-10 08:00:13,246 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/MasterData/WALs/jenkins-hbase4.apache.org,45287,1676016011592 2023-02-10 08:00:13,266 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45287%2C1676016011592, suffix=, logDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/MasterData/WALs/jenkins-hbase4.apache.org,45287,1676016011592, archiveDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/MasterData/oldWALs, maxLogs=10 2023-02-10 08:00:13,321 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43743,DS-e3349651-65d4-4752-b525-e3f72b4f516f,DISK] 2023-02-10 08:00:13,321 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40317,DS-dce3e251-5265-4dc8-924f-b4ded3491509,DISK] 2023-02-10 08:00:13,321 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40649,DS-280ee229-54a0-44df-8128-73e286576e3a,DISK] 2023-02-10 08:00:13,329 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-10 08:00:13,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/MasterData/WALs/jenkins-hbase4.apache.org,45287,1676016011592/jenkins-hbase4.apache.org%2C45287%2C1676016011592.1676016013276 2023-02-10 08:00:13,397 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40317,DS-dce3e251-5265-4dc8-924f-b4ded3491509,DISK], DatanodeInfoWithStorage[127.0.0.1:40649,DS-280ee229-54a0-44df-8128-73e286576e3a,DISK], DatanodeInfoWithStorage[127.0.0.1:43743,DS-e3349651-65d4-4752-b525-e3f72b4f516f,DISK]] 2023-02-10 08:00:13,398 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-10 08:00:13,398 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 08:00:13,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-10 08:00:13,403 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-10 08:00:13,460 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-10 08:00:13,468 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-10 08:00:13,491 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-10 08:00:13,503 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:13,508 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-10 08:00:13,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-10 08:00:13,523 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-10 08:00:13,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 08:00:13,528 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70085043, jitterRate=0.044348523020744324}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-10 08:00:13,528 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676016013428Initializing all the Stores at 1676016013431 (+3 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676016013431Cleaning up temporary data from old regions at 1676016013516 (+85 ms)Cleaning up detritus from prior splits at 1676016013517 (+1 ms)Region opened successfully at 1676016013528 (+11 ms) 2023-02-10 08:00:13,529 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-10 08:00:13,548 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-10 08:00:13,548 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-10 08:00:13,550 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 08:00:13,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-10 08:00:13,584 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 31 msec 2023-02-10 08:00:13,584 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-10 08:00:13,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-10 08:00:13,614 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-10 08:00:13,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-10 08:00:13,640 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-10 08:00:13,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-10 08:00:13,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-10 08:00:13,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-10 08:00:13,653 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:13,654 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-10 08:00:13,655 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-10 08:00:13,667 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-10 08:00:13,671 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-10 08:00:13,671 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-10 08:00:13,671 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-10 08:00:13,671 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-10 08:00:13,672 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:13,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,45287,1676016011592, sessionid=0x1003d3eafaf0000, setting cluster-up flag (Was=false) 2023-02-10 08:00:13,689 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:13,694 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-10 08:00:13,695 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45287,1676016011592 2023-02-10 08:00:13,701 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:13,707 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-10 08:00:13,708 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45287,1676016011592 2023-02-10 08:00:13,711 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/.hbase-snapshot/.tmp 2023-02-10 08:00:13,752 INFO [RS:1;jenkins-hbase4:34749] regionserver.HRegionServer(952): ClusterId : 855fac21-b430-4aac-843a-17f72c85fd80 2023-02-10 08:00:13,752 INFO [RS:0;jenkins-hbase4:36831] regionserver.HRegionServer(952): ClusterId : 855fac21-b430-4aac-843a-17f72c85fd80 2023-02-10 08:00:13,752 INFO [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(952): ClusterId : 855fac21-b430-4aac-843a-17f72c85fd80 2023-02-10 08:00:13,760 DEBUG [RS:1;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-10 08:00:13,760 DEBUG [RS:0;jenkins-hbase4:36831] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-10 08:00:13,760 DEBUG [RS:2;jenkins-hbase4:35917] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-10 08:00:13,766 DEBUG [RS:2;jenkins-hbase4:35917] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-10 08:00:13,766 DEBUG [RS:0;jenkins-hbase4:36831] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-10 08:00:13,766 DEBUG [RS:1;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-10 08:00:13,766 DEBUG [RS:0;jenkins-hbase4:36831] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-10 08:00:13,766 DEBUG [RS:2;jenkins-hbase4:35917] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-10 08:00:13,766 DEBUG [RS:1;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-10 08:00:13,771 DEBUG [RS:0;jenkins-hbase4:36831] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-10 08:00:13,771 DEBUG [RS:2;jenkins-hbase4:35917] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-10 08:00:13,772 DEBUG [RS:1;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-10 08:00:13,773 DEBUG [RS:0;jenkins-hbase4:36831] zookeeper.ReadOnlyZKClient(139): Connect 0x6b986e78 to 127.0.0.1:60578 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 08:00:13,774 DEBUG [RS:1;jenkins-hbase4:34749] zookeeper.ReadOnlyZKClient(139): Connect 0x2e7a8c53 to 127.0.0.1:60578 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 08:00:13,774 DEBUG [RS:2;jenkins-hbase4:35917] zookeeper.ReadOnlyZKClient(139): Connect 0x6a62bf71 to 127.0.0.1:60578 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 08:00:13,819 DEBUG [RS:0;jenkins-hbase4:36831] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@655f9d7b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 08:00:13,819 DEBUG [RS:2;jenkins-hbase4:35917] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@64a54c4d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 08:00:13,819 DEBUG [RS:0;jenkins-hbase4:36831] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@61600508, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-10 08:00:13,819 DEBUG [RS:1;jenkins-hbase4:34749] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@20bd77ec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 08:00:13,820 DEBUG [RS:2;jenkins-hbase4:35917] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@609ad401, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-10 08:00:13,820 DEBUG [RS:1;jenkins-hbase4:34749] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2482e723, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-10 08:00:13,841 DEBUG [RS:1;jenkins-hbase4:34749] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:34749 2023-02-10 08:00:13,841 DEBUG [RS:2;jenkins-hbase4:35917] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:35917 2023-02-10 08:00:13,845 INFO [RS:1;jenkins-hbase4:34749] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-10 08:00:13,846 INFO [RS:1;jenkins-hbase4:34749] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-10 08:00:13,845 INFO [RS:2;jenkins-hbase4:35917] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-10 08:00:13,846 INFO [RS:2;jenkins-hbase4:35917] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-10 08:00:13,846 DEBUG [RS:0;jenkins-hbase4:36831] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:36831 2023-02-10 08:00:13,846 DEBUG [RS:1;jenkins-hbase4:34749] regionserver.HRegionServer(1023): About to register with Master. 2023-02-10 08:00:13,846 INFO [RS:0;jenkins-hbase4:36831] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-10 08:00:13,846 DEBUG [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(1023): About to register with Master. 2023-02-10 08:00:13,846 INFO [RS:0;jenkins-hbase4:36831] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-10 08:00:13,846 DEBUG [RS:0;jenkins-hbase4:36831] regionserver.HRegionServer(1023): About to register with Master. 2023-02-10 08:00:13,849 INFO [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,45287,1676016011592 with isa=jenkins-hbase4.apache.org/172.31.14.131:35917, startcode=1676016012831 2023-02-10 08:00:13,849 INFO [RS:0;jenkins-hbase4:36831] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,45287,1676016011592 with isa=jenkins-hbase4.apache.org/172.31.14.131:36831, startcode=1676016012749 2023-02-10 08:00:13,849 INFO [RS:1;jenkins-hbase4:34749] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,45287,1676016011592 with isa=jenkins-hbase4.apache.org/172.31.14.131:34749, startcode=1676016012797 2023-02-10 08:00:13,862 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-10 08:00:13,865 DEBUG [RS:0;jenkins-hbase4:36831] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-10 08:00:13,865 DEBUG [RS:1;jenkins-hbase4:34749] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-10 08:00:13,865 DEBUG [RS:2;jenkins-hbase4:35917] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-10 08:00:13,871 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-10 08:00:13,871 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-10 08:00:13,871 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-10 08:00:13,871 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-10 08:00:13,872 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-10 08:00:13,872 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:13,872 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-10 08:00:13,872 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:13,873 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676016043873 2023-02-10 08:00:13,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-10 08:00:13,879 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-10 08:00:13,879 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-10 08:00:13,885 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-10 08:00:13,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-10 08:00:13,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-10 08:00:13,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-10 08:00:13,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-10 08:00:13,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-10 08:00:13,899 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:13,900 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-10 08:00:13,902 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-10 08:00:13,902 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-10 08:00:13,908 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-10 08:00:13,908 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-10 08:00:13,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676016013910,5,FailOnTimeoutGroup] 2023-02-10 08:00:13,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676016013912,5,FailOnTimeoutGroup] 2023-02-10 08:00:13,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:13,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-10 08:00:13,913 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:13,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:13,933 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49591, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-10 08:00:13,933 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43969, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-10 08:00:13,933 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56533, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-10 08:00:13,947 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-10 08:00:13,948 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-10 08:00:13,948 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12 2023-02-10 08:00:13,951 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45287] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36831,1676016012749 2023-02-10 08:00:13,951 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45287] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34749,1676016012797 2023-02-10 08:00:13,952 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45287] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:13,978 DEBUG [RS:1;jenkins-hbase4:34749] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12 2023-02-10 08:00:13,978 DEBUG [RS:0;jenkins-hbase4:36831] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12 2023-02-10 08:00:13,978 DEBUG [RS:0;jenkins-hbase4:36831] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:44445 2023-02-10 08:00:13,978 DEBUG [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12 2023-02-10 08:00:13,978 DEBUG [RS:0;jenkins-hbase4:36831] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-10 08:00:13,978 DEBUG [RS:1;jenkins-hbase4:34749] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:44445 2023-02-10 08:00:13,978 DEBUG [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:44445 2023-02-10 08:00:13,978 DEBUG [RS:1;jenkins-hbase4:34749] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-10 08:00:13,978 DEBUG [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-10 08:00:13,979 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 08:00:13,981 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-10 08:00:13,984 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/info 2023-02-10 08:00:13,985 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-10 08:00:13,985 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:13,986 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-10 08:00:13,987 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:13,987 DEBUG [RS:0;jenkins-hbase4:36831] zookeeper.ZKUtil(162): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36831,1676016012749 2023-02-10 08:00:13,988 DEBUG [RS:1;jenkins-hbase4:34749] zookeeper.ZKUtil(162): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34749,1676016012797 2023-02-10 08:00:13,988 WARN [RS:0;jenkins-hbase4:36831] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-10 08:00:13,988 WARN [RS:1;jenkins-hbase4:34749] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-10 08:00:13,989 INFO [RS:0;jenkins-hbase4:36831] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 08:00:13,989 INFO [RS:1;jenkins-hbase4:34749] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 08:00:13,989 DEBUG [RS:0;jenkins-hbase4:36831] regionserver.HRegionServer(1947): logDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/WALs/jenkins-hbase4.apache.org,36831,1676016012749 2023-02-10 08:00:13,989 DEBUG [RS:1;jenkins-hbase4:34749] regionserver.HRegionServer(1947): logDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/WALs/jenkins-hbase4.apache.org,34749,1676016012797 2023-02-10 08:00:13,989 DEBUG [RS:2;jenkins-hbase4:35917] zookeeper.ZKUtil(162): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:13,990 WARN [RS:2;jenkins-hbase4:35917] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-10 08:00:13,990 INFO [RS:2;jenkins-hbase4:35917] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 08:00:13,990 DEBUG [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(1947): logDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/WALs/jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:13,990 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34749,1676016012797] 2023-02-10 08:00:13,990 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36831,1676016012749] 2023-02-10 08:00:13,990 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,35917,1676016012831] 2023-02-10 08:00:13,991 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/rep_barrier 2023-02-10 08:00:13,996 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-10 08:00:13,999 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:13,999 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-10 08:00:14,002 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/table 2023-02-10 08:00:14,003 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-10 08:00:14,003 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:14,005 DEBUG [RS:1;jenkins-hbase4:34749] zookeeper.ZKUtil(162): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36831,1676016012749 2023-02-10 08:00:14,005 DEBUG [RS:0;jenkins-hbase4:36831] zookeeper.ZKUtil(162): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36831,1676016012749 2023-02-10 08:00:14,005 DEBUG [RS:2;jenkins-hbase4:35917] zookeeper.ZKUtil(162): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36831,1676016012749 2023-02-10 08:00:14,006 DEBUG [RS:1;jenkins-hbase4:34749] zookeeper.ZKUtil(162): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34749,1676016012797 2023-02-10 08:00:14,006 DEBUG [RS:0;jenkins-hbase4:36831] zookeeper.ZKUtil(162): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34749,1676016012797 2023-02-10 08:00:14,006 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740 2023-02-10 08:00:14,006 DEBUG [RS:2;jenkins-hbase4:35917] zookeeper.ZKUtil(162): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34749,1676016012797 2023-02-10 08:00:14,007 DEBUG [RS:1;jenkins-hbase4:34749] zookeeper.ZKUtil(162): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:14,007 DEBUG [RS:0;jenkins-hbase4:36831] zookeeper.ZKUtil(162): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:14,007 DEBUG [RS:2;jenkins-hbase4:35917] zookeeper.ZKUtil(162): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:14,007 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740 2023-02-10 08:00:14,014 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-10 08:00:14,016 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-10 08:00:14,017 DEBUG [RS:0;jenkins-hbase4:36831] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-10 08:00:14,017 DEBUG [RS:1;jenkins-hbase4:34749] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-10 08:00:14,017 DEBUG [RS:2;jenkins-hbase4:35917] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-10 08:00:14,021 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 08:00:14,022 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63335647, jitterRate=-0.05622531473636627}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-10 08:00:14,022 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676016013979Initializing all the Stores at 1676016013981 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676016013981Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676016013981Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676016013981Cleaning up temporary data from old regions at 1676016014012 (+31 ms)Cleaning up detritus from prior splits at 1676016014013 (+1 ms)Region opened successfully at 1676016014022 (+9 ms) 2023-02-10 08:00:14,022 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-10 08:00:14,023 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-10 08:00:14,023 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-10 08:00:14,023 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-10 08:00:14,023 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-10 08:00:14,027 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-10 08:00:14,028 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676016014022Disabling compacts and flushes for region at 1676016014022Disabling writes for close at 1676016014023 (+1 ms)Writing region close event to WAL at 1676016014027 (+4 ms)Closed at 1676016014027 2023-02-10 08:00:14,029 INFO [RS:1;jenkins-hbase4:34749] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-10 08:00:14,030 INFO [RS:0;jenkins-hbase4:36831] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-10 08:00:14,030 INFO [RS:2;jenkins-hbase4:35917] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-10 08:00:14,034 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-10 08:00:14,034 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-10 08:00:14,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-10 08:00:14,055 INFO [RS:0;jenkins-hbase4:36831] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-10 08:00:14,055 INFO [RS:1;jenkins-hbase4:34749] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-10 08:00:14,055 INFO [RS:2;jenkins-hbase4:35917] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-10 08:00:14,059 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-10 08:00:14,061 INFO [RS:2;jenkins-hbase4:35917] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-10 08:00:14,061 INFO [RS:0;jenkins-hbase4:36831] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-10 08:00:14,062 INFO [RS:2;jenkins-hbase4:35917] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,062 INFO [RS:0;jenkins-hbase4:36831] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,061 INFO [RS:1;jenkins-hbase4:34749] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-10 08:00:14,062 INFO [RS:1;jenkins-hbase4:34749] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,063 INFO [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-10 08:00:14,063 INFO [RS:0;jenkins-hbase4:36831] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-10 08:00:14,063 INFO [RS:1;jenkins-hbase4:34749] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-10 08:00:14,064 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-10 08:00:14,072 INFO [RS:2;jenkins-hbase4:35917] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,072 INFO [RS:0;jenkins-hbase4:36831] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,072 INFO [RS:1;jenkins-hbase4:34749] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,072 DEBUG [RS:2;jenkins-hbase4:35917] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:1;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:2;jenkins-hbase4:35917] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:1;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:2;jenkins-hbase4:35917] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:1;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,072 DEBUG [RS:0;jenkins-hbase4:36831] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:1;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:0;jenkins-hbase4:36831] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:1;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:0;jenkins-hbase4:36831] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:1;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-10 08:00:14,073 DEBUG [RS:0;jenkins-hbase4:36831] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:1;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:0;jenkins-hbase4:36831] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:2;jenkins-hbase4:35917] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:0;jenkins-hbase4:36831] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-10 08:00:14,073 DEBUG [RS:2;jenkins-hbase4:35917] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,074 DEBUG [RS:0;jenkins-hbase4:36831] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,074 DEBUG [RS:2;jenkins-hbase4:35917] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-10 08:00:14,074 DEBUG [RS:0;jenkins-hbase4:36831] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,074 DEBUG [RS:2;jenkins-hbase4:35917] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,074 DEBUG [RS:0;jenkins-hbase4:36831] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,074 DEBUG [RS:2;jenkins-hbase4:35917] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,074 DEBUG [RS:0;jenkins-hbase4:36831] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,074 DEBUG [RS:2;jenkins-hbase4:35917] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,073 DEBUG [RS:1;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,074 DEBUG [RS:2;jenkins-hbase4:35917] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,074 DEBUG [RS:1;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,075 DEBUG [RS:1;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:14,076 INFO [RS:0;jenkins-hbase4:36831] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,076 INFO [RS:0;jenkins-hbase4:36831] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,077 INFO [RS:2;jenkins-hbase4:35917] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,077 INFO [RS:1;jenkins-hbase4:34749] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,077 INFO [RS:0;jenkins-hbase4:36831] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,077 INFO [RS:1;jenkins-hbase4:34749] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,077 INFO [RS:2;jenkins-hbase4:35917] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,077 INFO [RS:1;jenkins-hbase4:34749] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,077 INFO [RS:2;jenkins-hbase4:35917] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,093 INFO [RS:0;jenkins-hbase4:36831] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-10 08:00:14,094 INFO [RS:1;jenkins-hbase4:34749] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-10 08:00:14,095 INFO [RS:2;jenkins-hbase4:35917] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-10 08:00:14,095 INFO [RS:2;jenkins-hbase4:35917] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35917,1676016012831-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,095 INFO [RS:0;jenkins-hbase4:36831] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36831,1676016012749-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,095 INFO [RS:1;jenkins-hbase4:34749] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34749,1676016012797-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,119 INFO [RS:2;jenkins-hbase4:35917] regionserver.Replication(203): jenkins-hbase4.apache.org,35917,1676016012831 started 2023-02-10 08:00:14,119 INFO [RS:0;jenkins-hbase4:36831] regionserver.Replication(203): jenkins-hbase4.apache.org,36831,1676016012749 started 2023-02-10 08:00:14,119 INFO [RS:1;jenkins-hbase4:34749] regionserver.Replication(203): jenkins-hbase4.apache.org,34749,1676016012797 started 2023-02-10 08:00:14,119 INFO [RS:0;jenkins-hbase4:36831] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,36831,1676016012749, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36831, sessionid=0x1003d3eafaf0001 2023-02-10 08:00:14,119 INFO [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,35917,1676016012831, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:35917, sessionid=0x1003d3eafaf0003 2023-02-10 08:00:14,119 INFO [RS:1;jenkins-hbase4:34749] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,34749,1676016012797, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34749, sessionid=0x1003d3eafaf0002 2023-02-10 08:00:14,120 DEBUG [RS:0;jenkins-hbase4:36831] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-10 08:00:14,120 DEBUG [RS:1;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-10 08:00:14,120 DEBUG [RS:2;jenkins-hbase4:35917] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-10 08:00:14,120 DEBUG [RS:1;jenkins-hbase4:34749] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34749,1676016012797 2023-02-10 08:00:14,120 DEBUG [RS:0;jenkins-hbase4:36831] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36831,1676016012749 2023-02-10 08:00:14,120 DEBUG [RS:1;jenkins-hbase4:34749] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34749,1676016012797' 2023-02-10 08:00:14,120 DEBUG [RS:2;jenkins-hbase4:35917] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:14,121 DEBUG [RS:1;jenkins-hbase4:34749] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 08:00:14,120 DEBUG [RS:0;jenkins-hbase4:36831] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36831,1676016012749' 2023-02-10 08:00:14,121 DEBUG [RS:0;jenkins-hbase4:36831] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 08:00:14,121 DEBUG [RS:2;jenkins-hbase4:35917] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35917,1676016012831' 2023-02-10 08:00:14,121 DEBUG [RS:2;jenkins-hbase4:35917] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 08:00:14,122 DEBUG [RS:1;jenkins-hbase4:34749] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 08:00:14,122 DEBUG [RS:2;jenkins-hbase4:35917] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 08:00:14,122 DEBUG [RS:0;jenkins-hbase4:36831] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 08:00:14,122 DEBUG [RS:1;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-10 08:00:14,122 DEBUG [RS:2;jenkins-hbase4:35917] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-10 08:00:14,123 DEBUG [RS:0;jenkins-hbase4:36831] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-10 08:00:14,123 DEBUG [RS:0;jenkins-hbase4:36831] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-10 08:00:14,123 DEBUG [RS:2;jenkins-hbase4:35917] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-10 08:00:14,122 DEBUG [RS:1;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-10 08:00:14,123 DEBUG [RS:1;jenkins-hbase4:34749] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34749,1676016012797 2023-02-10 08:00:14,123 DEBUG [RS:2;jenkins-hbase4:35917] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:14,123 DEBUG [RS:0;jenkins-hbase4:36831] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36831,1676016012749 2023-02-10 08:00:14,123 DEBUG [RS:0;jenkins-hbase4:36831] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36831,1676016012749' 2023-02-10 08:00:14,124 DEBUG [RS:0;jenkins-hbase4:36831] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-10 08:00:14,123 DEBUG [RS:2;jenkins-hbase4:35917] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35917,1676016012831' 2023-02-10 08:00:14,124 DEBUG [RS:2;jenkins-hbase4:35917] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-10 08:00:14,123 DEBUG [RS:1;jenkins-hbase4:34749] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34749,1676016012797' 2023-02-10 08:00:14,124 DEBUG [RS:1;jenkins-hbase4:34749] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-10 08:00:14,124 DEBUG [RS:0;jenkins-hbase4:36831] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-10 08:00:14,124 DEBUG [RS:2;jenkins-hbase4:35917] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-10 08:00:14,124 DEBUG [RS:1;jenkins-hbase4:34749] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-10 08:00:14,125 DEBUG [RS:0;jenkins-hbase4:36831] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-10 08:00:14,125 DEBUG [RS:2;jenkins-hbase4:35917] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-10 08:00:14,125 DEBUG [RS:1;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-10 08:00:14,125 INFO [RS:0;jenkins-hbase4:36831] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-10 08:00:14,125 INFO [RS:1;jenkins-hbase4:34749] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-10 08:00:14,125 INFO [RS:2;jenkins-hbase4:35917] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-10 08:00:14,126 INFO [RS:1;jenkins-hbase4:34749] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-10 08:00:14,125 INFO [RS:0;jenkins-hbase4:36831] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-10 08:00:14,126 INFO [RS:2;jenkins-hbase4:35917] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-10 08:00:14,216 DEBUG [jenkins-hbase4:45287] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-10 08:00:14,219 DEBUG [jenkins-hbase4:45287] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 08:00:14,225 DEBUG [jenkins-hbase4:45287] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 08:00:14,225 DEBUG [jenkins-hbase4:45287] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 08:00:14,225 DEBUG [jenkins-hbase4:45287] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 08:00:14,225 DEBUG [jenkins-hbase4:45287] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 08:00:14,228 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,35917,1676016012831, state=OPENING 2023-02-10 08:00:14,235 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-10 08:00:14,236 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:14,237 INFO [RS:0;jenkins-hbase4:36831] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36831%2C1676016012749, suffix=, logDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/WALs/jenkins-hbase4.apache.org,36831,1676016012749, archiveDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/oldWALs, maxLogs=32 2023-02-10 08:00:14,237 INFO [RS:2;jenkins-hbase4:35917] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35917%2C1676016012831, suffix=, logDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/WALs/jenkins-hbase4.apache.org,35917,1676016012831, archiveDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/oldWALs, maxLogs=32 2023-02-10 08:00:14,237 INFO [RS:1;jenkins-hbase4:34749] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34749%2C1676016012797, suffix=, logDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/WALs/jenkins-hbase4.apache.org,34749,1676016012797, archiveDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/oldWALs, maxLogs=32 2023-02-10 08:00:14,238 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-10 08:00:14,242 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,35917,1676016012831}] 2023-02-10 08:00:14,268 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40649,DS-280ee229-54a0-44df-8128-73e286576e3a,DISK] 2023-02-10 08:00:14,268 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40317,DS-dce3e251-5265-4dc8-924f-b4ded3491509,DISK] 2023-02-10 08:00:14,268 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40317,DS-dce3e251-5265-4dc8-924f-b4ded3491509,DISK] 2023-02-10 08:00:14,269 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43743,DS-e3349651-65d4-4752-b525-e3f72b4f516f,DISK] 2023-02-10 08:00:14,269 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40649,DS-280ee229-54a0-44df-8128-73e286576e3a,DISK] 2023-02-10 08:00:14,269 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43743,DS-e3349651-65d4-4752-b525-e3f72b4f516f,DISK] 2023-02-10 08:00:14,284 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40649,DS-280ee229-54a0-44df-8128-73e286576e3a,DISK] 2023-02-10 08:00:14,284 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43743,DS-e3349651-65d4-4752-b525-e3f72b4f516f,DISK] 2023-02-10 08:00:14,284 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40317,DS-dce3e251-5265-4dc8-924f-b4ded3491509,DISK] 2023-02-10 08:00:14,295 INFO [RS:0;jenkins-hbase4:36831] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/WALs/jenkins-hbase4.apache.org,36831,1676016012749/jenkins-hbase4.apache.org%2C36831%2C1676016012749.1676016014244 2023-02-10 08:00:14,295 INFO [RS:1;jenkins-hbase4:34749] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/WALs/jenkins-hbase4.apache.org,34749,1676016012797/jenkins-hbase4.apache.org%2C34749%2C1676016012797.1676016014244 2023-02-10 08:00:14,295 INFO [RS:2;jenkins-hbase4:35917] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/WALs/jenkins-hbase4.apache.org,35917,1676016012831/jenkins-hbase4.apache.org%2C35917%2C1676016012831.1676016014244 2023-02-10 08:00:14,295 DEBUG [RS:0;jenkins-hbase4:36831] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40317,DS-dce3e251-5265-4dc8-924f-b4ded3491509,DISK], DatanodeInfoWithStorage[127.0.0.1:43743,DS-e3349651-65d4-4752-b525-e3f72b4f516f,DISK], DatanodeInfoWithStorage[127.0.0.1:40649,DS-280ee229-54a0-44df-8128-73e286576e3a,DISK]] 2023-02-10 08:00:14,295 DEBUG [RS:1;jenkins-hbase4:34749] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40649,DS-280ee229-54a0-44df-8128-73e286576e3a,DISK], DatanodeInfoWithStorage[127.0.0.1:40317,DS-dce3e251-5265-4dc8-924f-b4ded3491509,DISK], DatanodeInfoWithStorage[127.0.0.1:43743,DS-e3349651-65d4-4752-b525-e3f72b4f516f,DISK]] 2023-02-10 08:00:14,295 DEBUG [RS:2;jenkins-hbase4:35917] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43743,DS-e3349651-65d4-4752-b525-e3f72b4f516f,DISK], DatanodeInfoWithStorage[127.0.0.1:40317,DS-dce3e251-5265-4dc8-924f-b4ded3491509,DISK], DatanodeInfoWithStorage[127.0.0.1:40649,DS-280ee229-54a0-44df-8128-73e286576e3a,DISK]] 2023-02-10 08:00:14,434 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:14,436 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-10 08:00:14,439 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37590, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-10 08:00:14,452 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-10 08:00:14,453 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 08:00:14,457 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35917%2C1676016012831.meta, suffix=.meta, logDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/WALs/jenkins-hbase4.apache.org,35917,1676016012831, archiveDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/oldWALs, maxLogs=32 2023-02-10 08:00:14,478 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43743,DS-e3349651-65d4-4752-b525-e3f72b4f516f,DISK] 2023-02-10 08:00:14,478 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40317,DS-dce3e251-5265-4dc8-924f-b4ded3491509,DISK] 2023-02-10 08:00:14,481 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40649,DS-280ee229-54a0-44df-8128-73e286576e3a,DISK] 2023-02-10 08:00:14,486 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/WALs/jenkins-hbase4.apache.org,35917,1676016012831/jenkins-hbase4.apache.org%2C35917%2C1676016012831.meta.1676016014458.meta 2023-02-10 08:00:14,486 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40317,DS-dce3e251-5265-4dc8-924f-b4ded3491509,DISK], DatanodeInfoWithStorage[127.0.0.1:43743,DS-e3349651-65d4-4752-b525-e3f72b4f516f,DISK], DatanodeInfoWithStorage[127.0.0.1:40649,DS-280ee229-54a0-44df-8128-73e286576e3a,DISK]] 2023-02-10 08:00:14,486 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-10 08:00:14,490 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-10 08:00:14,506 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-10 08:00:14,511 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-10 08:00:14,517 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-10 08:00:14,517 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 08:00:14,517 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-10 08:00:14,517 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-10 08:00:14,520 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-10 08:00:14,522 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/info 2023-02-10 08:00:14,522 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/info 2023-02-10 08:00:14,524 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-10 08:00:14,526 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:14,527 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-10 08:00:14,530 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/rep_barrier 2023-02-10 08:00:14,530 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/rep_barrier 2023-02-10 08:00:14,531 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-10 08:00:14,532 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:14,532 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-10 08:00:14,533 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/table 2023-02-10 08:00:14,533 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/table 2023-02-10 08:00:14,534 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-10 08:00:14,535 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:14,536 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740 2023-02-10 08:00:14,539 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740 2023-02-10 08:00:14,543 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-10 08:00:14,545 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-10 08:00:14,546 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67808574, jitterRate=0.010426491498947144}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-10 08:00:14,547 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676016014517Writing region info on filesystem at 1676016014517Initializing all the Stores at 1676016014519 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676016014520 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676016014520Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676016014520Cleaning up temporary data from old regions at 1676016014541 (+21 ms)Cleaning up detritus from prior splits at 1676016014541Running coprocessor post-open hooks at 1676016014546 (+5 ms)Region opened successfully at 1676016014547 (+1 ms) 2023-02-10 08:00:14,556 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676016014426 2023-02-10 08:00:14,573 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-10 08:00:14,574 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-10 08:00:14,575 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,35917,1676016012831, state=OPEN 2023-02-10 08:00:14,578 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-10 08:00:14,578 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-10 08:00:14,582 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-10 08:00:14,582 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,35917,1676016012831 in 336 msec 2023-02-10 08:00:14,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-10 08:00:14,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 540 msec 2023-02-10 08:00:14,595 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 841 msec 2023-02-10 08:00:14,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676016012875, completionTime=-1 2023-02-10 08:00:14,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-10 08:00:14,595 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 08:00:14,662 DEBUG [hconnection-0x692be1ea-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-10 08:00:14,665 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37596, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-10 08:00:14,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 08:00:14,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676016074683 2023-02-10 08:00:14,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676016134683 2023-02-10 08:00:14,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 87 msec 2023-02-10 08:00:14,708 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45287,1676016011592-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,708 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45287,1676016011592-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,708 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45287,1676016011592-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,710 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:45287, period=300000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,710 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:14,720 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-10 08:00:14,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-10 08:00:14,730 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-10 08:00:14,742 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-10 08:00:14,744 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-10 08:00:14,747 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-10 08:00:14,769 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/.tmp/data/hbase/namespace/90e1994fa126850d3f7b02ee5eec2939 2023-02-10 08:00:14,772 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/.tmp/data/hbase/namespace/90e1994fa126850d3f7b02ee5eec2939 empty. 2023-02-10 08:00:14,772 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/.tmp/data/hbase/namespace/90e1994fa126850d3f7b02ee5eec2939 2023-02-10 08:00:14,773 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-10 08:00:14,808 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-10 08:00:14,811 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 90e1994fa126850d3f7b02ee5eec2939, NAME => 'hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/.tmp 2023-02-10 08:00:14,829 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 08:00:14,829 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 90e1994fa126850d3f7b02ee5eec2939, disabling compactions & flushes 2023-02-10 08:00:14,829 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. 2023-02-10 08:00:14,829 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. 2023-02-10 08:00:14,829 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. after waiting 0 ms 2023-02-10 08:00:14,829 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. 2023-02-10 08:00:14,829 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. 2023-02-10 08:00:14,829 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 90e1994fa126850d3f7b02ee5eec2939: Waiting for close lock at 1676016014829Disabling compacts and flushes for region at 1676016014829Disabling writes for close at 1676016014829Writing region close event to WAL at 1676016014829Closed at 1676016014829 2023-02-10 08:00:14,834 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-10 08:00:14,851 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676016014838"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676016014838"}]},"ts":"1676016014838"} 2023-02-10 08:00:14,878 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-10 08:00:14,880 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-10 08:00:14,885 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676016014880"}]},"ts":"1676016014880"} 2023-02-10 08:00:14,889 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-10 08:00:14,893 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 08:00:14,895 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 08:00:14,895 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 08:00:14,895 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 08:00:14,895 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 08:00:14,897 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=90e1994fa126850d3f7b02ee5eec2939, ASSIGN}] 2023-02-10 08:00:14,900 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=90e1994fa126850d3f7b02ee5eec2939, ASSIGN 2023-02-10 08:00:14,902 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=90e1994fa126850d3f7b02ee5eec2939, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35917,1676016012831; forceNewPlan=false, retain=false 2023-02-10 08:00:15,055 INFO [jenkins-hbase4:45287] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 08:00:15,056 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=90e1994fa126850d3f7b02ee5eec2939, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:15,056 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676016015056"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676016015056"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676016015056"}]},"ts":"1676016015056"} 2023-02-10 08:00:15,060 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 90e1994fa126850d3f7b02ee5eec2939, server=jenkins-hbase4.apache.org,35917,1676016012831}] 2023-02-10 08:00:15,223 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. 2023-02-10 08:00:15,224 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 90e1994fa126850d3f7b02ee5eec2939, NAME => 'hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939.', STARTKEY => '', ENDKEY => ''} 2023-02-10 08:00:15,224 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 90e1994fa126850d3f7b02ee5eec2939 2023-02-10 08:00:15,224 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 08:00:15,225 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 90e1994fa126850d3f7b02ee5eec2939 2023-02-10 08:00:15,225 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 90e1994fa126850d3f7b02ee5eec2939 2023-02-10 08:00:15,227 INFO [StoreOpener-90e1994fa126850d3f7b02ee5eec2939-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 90e1994fa126850d3f7b02ee5eec2939 2023-02-10 08:00:15,229 DEBUG [StoreOpener-90e1994fa126850d3f7b02ee5eec2939-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/namespace/90e1994fa126850d3f7b02ee5eec2939/info 2023-02-10 08:00:15,229 DEBUG [StoreOpener-90e1994fa126850d3f7b02ee5eec2939-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/namespace/90e1994fa126850d3f7b02ee5eec2939/info 2023-02-10 08:00:15,229 INFO [StoreOpener-90e1994fa126850d3f7b02ee5eec2939-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 90e1994fa126850d3f7b02ee5eec2939 columnFamilyName info 2023-02-10 08:00:15,230 INFO [StoreOpener-90e1994fa126850d3f7b02ee5eec2939-1] regionserver.HStore(310): Store=90e1994fa126850d3f7b02ee5eec2939/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:15,232 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/namespace/90e1994fa126850d3f7b02ee5eec2939 2023-02-10 08:00:15,233 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/namespace/90e1994fa126850d3f7b02ee5eec2939 2023-02-10 08:00:15,237 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 90e1994fa126850d3f7b02ee5eec2939 2023-02-10 08:00:15,241 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/namespace/90e1994fa126850d3f7b02ee5eec2939/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 08:00:15,241 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 90e1994fa126850d3f7b02ee5eec2939; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70951668, jitterRate=0.05726224184036255}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-10 08:00:15,241 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 90e1994fa126850d3f7b02ee5eec2939: Running coprocessor pre-open hook at 1676016015225Writing region info on filesystem at 1676016015225Initializing all the Stores at 1676016015226 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676016015226Cleaning up temporary data from old regions at 1676016015233 (+7 ms)Cleaning up detritus from prior splits at 1676016015234 (+1 ms)Running coprocessor post-open hooks at 1676016015241 (+7 ms)Region opened successfully at 1676016015241 2023-02-10 08:00:15,244 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939., pid=6, masterSystemTime=1676016015214 2023-02-10 08:00:15,247 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. 2023-02-10 08:00:15,247 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. 2023-02-10 08:00:15,249 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=90e1994fa126850d3f7b02ee5eec2939, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:15,250 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676016015248"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676016015248"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676016015248"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676016015248"}]},"ts":"1676016015248"} 2023-02-10 08:00:15,256 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-10 08:00:15,256 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 90e1994fa126850d3f7b02ee5eec2939, server=jenkins-hbase4.apache.org,35917,1676016012831 in 193 msec 2023-02-10 08:00:15,261 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-10 08:00:15,261 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=90e1994fa126850d3f7b02ee5eec2939, ASSIGN in 359 msec 2023-02-10 08:00:15,262 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-10 08:00:15,262 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676016015262"}]},"ts":"1676016015262"} 2023-02-10 08:00:15,265 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-10 08:00:15,269 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-10 08:00:15,271 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 538 msec 2023-02-10 08:00:15,345 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-10 08:00:15,346 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-10 08:00:15,346 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:15,381 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-10 08:00:15,432 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-10 08:00:15,438 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 64 msec 2023-02-10 08:00:15,446 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-10 08:00:15,457 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-10 08:00:15,464 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-02-10 08:00:15,472 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-10 08:00:15,475 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-10 08:00:15,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.590sec 2023-02-10 08:00:15,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-10 08:00:15,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-10 08:00:15,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-10 08:00:15,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45287,1676016011592-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-10 08:00:15,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45287,1676016011592-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-10 08:00:15,497 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-10 08:00:15,558 DEBUG [Listener at localhost/33663] zookeeper.ReadOnlyZKClient(139): Connect 0x64f88aaa to 127.0.0.1:60578 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 08:00:15,564 DEBUG [Listener at localhost/33663] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5e2a2984, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 08:00:15,578 DEBUG [hconnection-0x1f9a07a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-10 08:00:15,590 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37610, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-10 08:00:15,600 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,45287,1676016011592 2023-02-10 08:00:15,600 DEBUG [Listener at localhost/33663] zookeeper.ReadOnlyZKClient(139): Connect 0x773af542 to 127.0.0.1:60578 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 08:00:15,611 DEBUG [ReadOnlyZKClient-127.0.0.1:60578@0x773af542] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d939af, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 08:00:15,645 DEBUG [Listener at localhost/33663] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-10 08:00:15,649 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39190, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-10 08:00:15,650 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36831] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,36831,1676016012749' ***** 2023-02-10 08:00:15,650 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36831] regionserver.HRegionServer(2310): STOPPED: Called by admin client org.apache.hadoop.hbase.client.AsyncConnectionImpl@2164f666 2023-02-10 08:00:15,652 INFO [RS:0;jenkins-hbase4:36831] regionserver.HeapMemoryManager(220): Stopping 2023-02-10 08:00:15,652 INFO [RS:0;jenkins-hbase4:36831] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-10 08:00:15,652 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-10 08:00:15,652 INFO [RS:0;jenkins-hbase4:36831] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-10 08:00:15,653 INFO [RS:0;jenkins-hbase4:36831] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,36831,1676016012749 2023-02-10 08:00:15,653 DEBUG [RS:0;jenkins-hbase4:36831] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6b986e78 to 127.0.0.1:60578 2023-02-10 08:00:15,653 DEBUG [RS:0;jenkins-hbase4:36831] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:15,653 INFO [RS:0;jenkins-hbase4:36831] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,36831,1676016012749; all regions closed. 2023-02-10 08:00:15,657 DEBUG [Listener at localhost/33663] client.ConnectionUtils(586): Start fetching master stub from registry 2023-02-10 08:00:15,660 DEBUG [ReadOnlyZKClient-127.0.0.1:60578@0x773af542] client.AsyncConnectionImpl(289): The fetched master address is jenkins-hbase4.apache.org,45287,1676016011592 2023-02-10 08:00:15,663 DEBUG [ReadOnlyZKClient-127.0.0.1:60578@0x773af542] client.ConnectionUtils(594): The fetched master stub is org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$Stub@60962eae 2023-02-10 08:00:15,669 DEBUG [ReadOnlyZKClient-127.0.0.1:60578@0x773af542] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-10 08:00:15,671 DEBUG [RS:0;jenkins-hbase4:36831] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/oldWALs 2023-02-10 08:00:15,672 INFO [RS:0;jenkins-hbase4:36831] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36831%2C1676016012749:(num 1676016014244) 2023-02-10 08:00:15,672 DEBUG [RS:0;jenkins-hbase4:36831] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:15,673 INFO [RS:0;jenkins-hbase4:36831] regionserver.LeaseManager(133): Closed leases 2023-02-10 08:00:15,675 INFO [RS:0;jenkins-hbase4:36831] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-10 08:00:15,675 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58450, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-10 08:00:15,675 INFO [RS:0;jenkins-hbase4:36831] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-10 08:00:15,675 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-10 08:00:15,675 INFO [RS:0;jenkins-hbase4:36831] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-10 08:00:15,676 INFO [RS:0;jenkins-hbase4:36831] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-10 08:00:15,676 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45287] master.MasterRpcServices(1601): Client=jenkins//172.31.14.131 stop 2023-02-10 08:00:15,676 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45287] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,45287,1676016011592' ***** 2023-02-10 08:00:15,676 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45287] regionserver.HRegionServer(2310): STOPPED: Stopped by RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45287 2023-02-10 08:00:15,677 DEBUG [M:0;jenkins-hbase4:45287] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4921781c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-10 08:00:15,677 INFO [RS:0;jenkins-hbase4:36831] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36831 2023-02-10 08:00:15,677 INFO [M:0;jenkins-hbase4:45287] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,45287,1676016011592 2023-02-10 08:00:15,677 DEBUG [M:0;jenkins-hbase4:45287] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2e677d8b to 127.0.0.1:60578 2023-02-10 08:00:15,677 DEBUG [M:0;jenkins-hbase4:45287] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:15,678 INFO [M:0;jenkins-hbase4:45287] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,45287,1676016011592; all regions closed. 2023-02-10 08:00:15,678 DEBUG [M:0;jenkins-hbase4:45287] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:15,678 DEBUG [M:0;jenkins-hbase4:45287] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-10 08:00:15,678 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-10 08:00:15,678 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676016013912] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676016013912,5,FailOnTimeoutGroup] 2023-02-10 08:00:15,678 DEBUG [M:0;jenkins-hbase4:45287] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-10 08:00:15,678 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676016013910] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676016013910,5,FailOnTimeoutGroup] 2023-02-10 08:00:15,679 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-10 08:00:15,679 INFO [M:0;jenkins-hbase4:45287] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-10 08:00:15,680 INFO [M:0;jenkins-hbase4:45287] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-10 08:00:15,680 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-10 08:00:15,680 INFO [M:0;jenkins-hbase4:45287] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-10 08:00:15,680 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-10 08:00:15,681 DEBUG [M:0;jenkins-hbase4:45287] master.HMaster(1502): Stopping service threads 2023-02-10 08:00:15,681 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-10 08:00:15,681 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-10 08:00:15,681 INFO [M:0;jenkins-hbase4:45287] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-10 08:00:15,681 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:15,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 08:00:15,683 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 08:00:15,683 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 08:00:15,683 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 08:00:15,684 INFO [M:0;jenkins-hbase4:45287] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 08:00:15,684 INFO [Listener at localhost/33663] client.AsyncConnectionImpl(207): Connection has been closed by Listener at localhost/33663. 2023-02-10 08:00:15,685 DEBUG [Listener at localhost/33663] client.AsyncConnectionImpl(232): Call stack: at java.lang.Thread.getStackTrace(Thread.java:1564) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:209) at org.apache.hbase.thirdparty.com.google.common.io.Closeables.close(Closeables.java:79) at org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2.tearDown(TestAsyncClusterAdminApi2.java:75) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2023-02-10 08:00:15,688 ERROR [M:0;jenkins-hbase4:45287] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] 2023-02-10 08:00:15,689 INFO [M:0;jenkins-hbase4:45287] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 08:00:15,689 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-10 08:00:15,690 DEBUG [M:0;jenkins-hbase4:45287] zookeeper.ZKUtil(398): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-10 08:00:15,690 WARN [M:0;jenkins-hbase4:45287] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-10 08:00:15,690 INFO [M:0;jenkins-hbase4:45287] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 08:00:15,691 INFO [M:0;jenkins-hbase4:45287] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-10 08:00:15,691 DEBUG [M:0;jenkins-hbase4:45287] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-10 08:00:15,691 INFO [M:0;jenkins-hbase4:45287] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:15,691 DEBUG [M:0;jenkins-hbase4:45287] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:15,691 DEBUG [M:0;jenkins-hbase4:45287] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-10 08:00:15,691 DEBUG [M:0;jenkins-hbase4:45287] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:15,694 INFO [M:0;jenkins-hbase4:45287] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-02-10 08:00:15,695 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36831,1676016012749 2023-02-10 08:00:15,695 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36831,1676016012749 2023-02-10 08:00:15,695 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36831,1676016012749 2023-02-10 08:00:15,695 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:15,695 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:15,695 ERROR [Listener at localhost/33663-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4220683 rejected from java.util.concurrent.ThreadPoolExecutor@1d92df[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:15,695 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:15,695 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:15,695 ERROR [Listener at localhost/33663-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6012bd35 rejected from java.util.concurrent.ThreadPoolExecutor@1d92df[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:15,696 DEBUG [Listener at localhost/33663] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:15,697 DEBUG [Listener at localhost/33663] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x773af542 to 127.0.0.1:60578 2023-02-10 08:00:15,697 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-10 08:00:15,697 DEBUG [Listener at localhost/33663] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x64f88aaa to 127.0.0.1:60578 2023-02-10 08:00:15,697 DEBUG [Listener at localhost/33663] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:15,698 DEBUG [Listener at localhost/33663] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-10 08:00:15,698 INFO [Listener at localhost/33663] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,34749,1676016012797' ***** 2023-02-10 08:00:15,698 INFO [Listener at localhost/33663] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-10 08:00:15,698 INFO [Listener at localhost/33663] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,35917,1676016012831' ***** 2023-02-10 08:00:15,699 INFO [Listener at localhost/33663] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-10 08:00:15,699 INFO [RS:1;jenkins-hbase4:34749] regionserver.HeapMemoryManager(220): Stopping 2023-02-10 08:00:15,699 INFO [RS:2;jenkins-hbase4:35917] regionserver.HeapMemoryManager(220): Stopping 2023-02-10 08:00:15,699 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-10 08:00:15,699 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-10 08:00:15,699 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34749,1676016012797 2023-02-10 08:00:15,698 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34749,1676016012797 2023-02-10 08:00:15,699 INFO [RS:2;jenkins-hbase4:35917] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-10 08:00:15,700 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:15,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:15,701 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,36831,1676016012749 znode expired, triggering replicatorRemoved event 2023-02-10 08:00:15,699 INFO [RS:1;jenkins-hbase4:34749] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-10 08:00:15,702 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,36831,1676016012749 znode expired, triggering replicatorRemoved event 2023-02-10 08:00:15,700 INFO [RS:2;jenkins-hbase4:35917] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-10 08:00:15,702 INFO [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(3304): Received CLOSE for 90e1994fa126850d3f7b02ee5eec2939 2023-02-10 08:00:15,702 INFO [RS:1;jenkins-hbase4:34749] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-10 08:00:15,703 INFO [RS:1;jenkins-hbase4:34749] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,34749,1676016012797 2023-02-10 08:00:15,703 INFO [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:15,703 DEBUG [RS:1;jenkins-hbase4:34749] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2e7a8c53 to 127.0.0.1:60578 2023-02-10 08:00:15,703 DEBUG [RS:1;jenkins-hbase4:34749] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:15,703 INFO [RS:1;jenkins-hbase4:34749] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,34749,1676016012797; all regions closed. 2023-02-10 08:00:15,703 DEBUG [RS:2;jenkins-hbase4:35917] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6a62bf71 to 127.0.0.1:60578 2023-02-10 08:00:15,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 90e1994fa126850d3f7b02ee5eec2939, disabling compactions & flushes 2023-02-10 08:00:15,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. 2023-02-10 08:00:15,704 DEBUG [RS:2;jenkins-hbase4:35917] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:15,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. 2023-02-10 08:00:15,704 INFO [RS:2;jenkins-hbase4:35917] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-10 08:00:15,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. after waiting 0 ms 2023-02-10 08:00:15,704 INFO [RS:2;jenkins-hbase4:35917] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-10 08:00:15,704 INFO [RS:2;jenkins-hbase4:35917] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-10 08:00:15,705 INFO [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-10 08:00:15,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. 2023-02-10 08:00:15,705 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 90e1994fa126850d3f7b02ee5eec2939 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-10 08:00:15,705 INFO [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-02-10 08:00:15,706 DEBUG [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740, 90e1994fa126850d3f7b02ee5eec2939=hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939.} 2023-02-10 08:00:15,706 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-10 08:00:15,706 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-10 08:00:15,706 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-10 08:00:15,706 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-10 08:00:15,706 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-10 08:00:15,707 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-02-10 08:00:15,709 DEBUG [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(1505): Waiting on 1588230740, 90e1994fa126850d3f7b02ee5eec2939 2023-02-10 08:00:15,715 DEBUG [RS:1;jenkins-hbase4:34749] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/oldWALs 2023-02-10 08:00:15,715 INFO [RS:1;jenkins-hbase4:34749] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34749%2C1676016012797:(num 1676016014244) 2023-02-10 08:00:15,715 DEBUG [RS:1;jenkins-hbase4:34749] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:15,715 INFO [RS:1;jenkins-hbase4:34749] regionserver.LeaseManager(133): Closed leases 2023-02-10 08:00:15,716 INFO [RS:1;jenkins-hbase4:34749] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-10 08:00:15,716 INFO [RS:1;jenkins-hbase4:34749] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-10 08:00:15,716 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-10 08:00:15,716 INFO [RS:1;jenkins-hbase4:34749] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-10 08:00:15,716 INFO [RS:1;jenkins-hbase4:34749] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-10 08:00:15,717 INFO [RS:1;jenkins-hbase4:34749] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34749 2023-02-10 08:00:15,720 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34749,1676016012797 2023-02-10 08:00:15,720 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34749,1676016012797 2023-02-10 08:00:15,721 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:15,721 ERROR [Listener at localhost/33663-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5db590d2 rejected from java.util.concurrent.ThreadPoolExecutor@598a1fbc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:15,721 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:15,721 ERROR [Listener at localhost/33663-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5c1c59a0 rejected from java.util.concurrent.ThreadPoolExecutor@598a1fbc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:15,779 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-10 08:00:15,779 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-10 08:00:15,787 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/.tmp/info/fddfbeda1de640188bc902b38a000c19 2023-02-10 08:00:15,788 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/namespace/90e1994fa126850d3f7b02ee5eec2939/.tmp/info/bafc897a814b4f58ac86e598625bf8c6 2023-02-10 08:00:15,789 INFO [M:0;jenkins-hbase4:45287] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ff1c662002ec47828bb1697d0efac52e 2023-02-10 08:00:15,829 DEBUG [M:0;jenkins-hbase4:45287] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ff1c662002ec47828bb1697d0efac52e as hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ff1c662002ec47828bb1697d0efac52e 2023-02-10 08:00:15,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/namespace/90e1994fa126850d3f7b02ee5eec2939/.tmp/info/bafc897a814b4f58ac86e598625bf8c6 as hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/namespace/90e1994fa126850d3f7b02ee5eec2939/info/bafc897a814b4f58ac86e598625bf8c6 2023-02-10 08:00:15,839 INFO [M:0;jenkins-hbase4:45287] regionserver.HStore(1080): Added hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ff1c662002ec47828bb1697d0efac52e, entries=8, sequenceid=66, filesize=6.3 K 2023-02-10 08:00:15,841 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/namespace/90e1994fa126850d3f7b02ee5eec2939/info/bafc897a814b4f58ac86e598625bf8c6, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 08:00:15,841 INFO [M:0;jenkins-hbase4:45287] regionserver.HRegion(2947): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 147ms, sequenceid=66, compaction requested=false 2023-02-10 08:00:15,845 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 90e1994fa126850d3f7b02ee5eec2939 in 140ms, sequenceid=6, compaction requested=false 2023-02-10 08:00:15,845 INFO [M:0;jenkins-hbase4:45287] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:15,846 DEBUG [M:0;jenkins-hbase4:45287] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676016015691Disabling compacts and flushes for region at 1676016015691Disabling writes for close at 1676016015691Obtaining lock to block concurrent updates at 1676016015694 (+3 ms)Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676016015694Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=24646, getHeapSize=30248, getOffHeapSize=0, getCellsCount=71 at 1676016015712 (+18 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676016015714 (+2 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676016015717 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676016015747 (+30 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676016015750 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676016015831 (+81 ms)Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 147ms, sequenceid=66, compaction requested=false at 1676016015841 (+10 ms)Writing region close event to WAL at 1676016015845 (+4 ms)Closed at 1676016015845 2023-02-10 08:00:15,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-10 08:00:15,858 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:15,858 INFO [RS:1;jenkins-hbase4:34749] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,34749,1676016012797; zookeeper connection closed. 2023-02-10 08:00:15,859 ERROR [Listener at localhost/33663-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@24a0fc09 rejected from java.util.concurrent.ThreadPoolExecutor@598a1fbc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:15,859 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-10 08:00:15,859 INFO [M:0;jenkins-hbase4:45287] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-10 08:00:15,859 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:34749-0x1003d3eafaf0002, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:15,859 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@759a0bef] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@759a0bef 2023-02-10 08:00:15,859 ERROR [Listener at localhost/33663-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@42c8f2cb rejected from java.util.concurrent.ThreadPoolExecutor@598a1fbc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:15,862 INFO [M:0;jenkins-hbase4:45287] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45287 2023-02-10 08:00:15,867 DEBUG [M:0;jenkins-hbase4:45287] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,45287,1676016011592 already deleted, retry=false 2023-02-10 08:00:15,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/namespace/90e1994fa126850d3f7b02ee5eec2939/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-10 08:00:15,872 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. 2023-02-10 08:00:15,872 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/.tmp/table/41deec0f13224cbb894aec89a4e7ecf4 2023-02-10 08:00:15,872 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 90e1994fa126850d3f7b02ee5eec2939: Waiting for close lock at 1676016015703Running coprocessor pre-close hooks at 1676016015703Disabling compacts and flushes for region at 1676016015703Disabling writes for close at 1676016015704 (+1 ms)Obtaining lock to block concurrent updates at 1676016015705 (+1 ms)Preparing flush snapshotting stores in 90e1994fa126850d3f7b02ee5eec2939 at 1676016015705Finished memstore snapshotting hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676016015712 (+7 ms)Flushing stores of hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. at 1676016015714 (+2 ms)Flushing 90e1994fa126850d3f7b02ee5eec2939/info: creating writer at 1676016015717 (+3 ms)Flushing 90e1994fa126850d3f7b02ee5eec2939/info: appending metadata at 1676016015747 (+30 ms)Flushing 90e1994fa126850d3f7b02ee5eec2939/info: closing flushed file at 1676016015750 (+3 ms)Flushing 90e1994fa126850d3f7b02ee5eec2939/info: reopening flushed file at 1676016015831 (+81 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 90e1994fa126850d3f7b02ee5eec2939 in 140ms, sequenceid=6, compaction requested=false at 1676016015845 (+14 ms)Writing region close event to WAL at 1676016015857 (+12 ms)Running coprocessor post-close hooks at 1676016015872 (+15 ms)Closed at 1676016015872 2023-02-10 08:00:15,872 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676016014729.90e1994fa126850d3f7b02ee5eec2939. 2023-02-10 08:00:15,881 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/.tmp/info/fddfbeda1de640188bc902b38a000c19 as hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/info/fddfbeda1de640188bc902b38a000c19 2023-02-10 08:00:15,891 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/info/fddfbeda1de640188bc902b38a000c19, entries=10, sequenceid=9, filesize=5.9 K 2023-02-10 08:00:15,893 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/.tmp/table/41deec0f13224cbb894aec89a4e7ecf4 as hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/table/41deec0f13224cbb894aec89a4e7ecf4 2023-02-10 08:00:15,903 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/table/41deec0f13224cbb894aec89a4e7ecf4, entries=2, sequenceid=9, filesize=4.7 K 2023-02-10 08:00:15,905 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 198ms, sequenceid=9, compaction requested=false 2023-02-10 08:00:15,905 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-10 08:00:15,909 DEBUG [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-10 08:00:15,914 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-02-10 08:00:15,914 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-10 08:00:15,915 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-10 08:00:15,915 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676016015706Running coprocessor pre-close hooks at 1676016015706Disabling compacts and flushes for region at 1676016015706Disabling writes for close at 1676016015706Obtaining lock to block concurrent updates at 1676016015708 (+2 ms)Preparing flush snapshotting stores in 1588230740 at 1676016015708Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=1290, getHeapSize=2912, getOffHeapSize=0, getCellsCount=12 at 1676016015712 (+4 ms)Flushing stores of hbase:meta,,1.1588230740 at 1676016015718 (+6 ms)Flushing 1588230740/info: creating writer at 1676016015720 (+2 ms)Flushing 1588230740/info: appending metadata at 1676016015746 (+26 ms)Flushing 1588230740/info: closing flushed file at 1676016015750 (+4 ms)Flushing 1588230740/table: creating writer at 1676016015827 (+77 ms)Flushing 1588230740/table: appending metadata at 1676016015843 (+16 ms)Flushing 1588230740/table: closing flushed file at 1676016015843Flushing 1588230740/info: reopening flushed file at 1676016015883 (+40 ms)Flushing 1588230740/table: reopening flushed file at 1676016015894 (+11 ms)Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 198ms, sequenceid=9, compaction requested=false at 1676016015905 (+11 ms)Writing region close event to WAL at 1676016015909 (+4 ms)Running coprocessor post-close hooks at 1676016015914 (+5 ms)Closed at 1676016015915 (+1 ms) 2023-02-10 08:00:15,915 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-10 08:00:16,082 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-10 08:00:16,082 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-10 08:00:16,110 INFO [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,35917,1676016012831; all regions closed. 2023-02-10 08:00:16,117 DEBUG [RS:2;jenkins-hbase4:35917] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/oldWALs 2023-02-10 08:00:16,117 INFO [RS:2;jenkins-hbase4:35917] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35917%2C1676016012831.meta:.meta(num 1676016014458) 2023-02-10 08:00:16,123 DEBUG [RS:2;jenkins-hbase4:35917] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/oldWALs 2023-02-10 08:00:16,123 INFO [RS:2;jenkins-hbase4:35917] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35917%2C1676016012831:(num 1676016014244) 2023-02-10 08:00:16,123 DEBUG [RS:2;jenkins-hbase4:35917] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:16,123 INFO [RS:2;jenkins-hbase4:35917] regionserver.LeaseManager(133): Closed leases 2023-02-10 08:00:16,123 INFO [RS:2;jenkins-hbase4:35917] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-10 08:00:16,124 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-10 08:00:16,124 INFO [RS:2;jenkins-hbase4:35917] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35917 2023-02-10 08:00:16,127 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,35917,1676016012831 2023-02-10 08:00:16,127 ERROR [Listener at localhost/33663-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@58eb2727 rejected from java.util.concurrent.ThreadPoolExecutor@5cd4b8e2[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:16,359 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:16,359 INFO [RS:0;jenkins-hbase4:36831] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,36831,1676016012749; zookeeper connection closed. 2023-02-10 08:00:16,359 ERROR [Listener at localhost/33663-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1877468d rejected from java.util.concurrent.ThreadPoolExecutor@1d92df[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:16,359 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:36831-0x1003d3eafaf0001, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:16,359 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@51e1b2be] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@51e1b2be 2023-02-10 08:00:16,359 ERROR [Listener at localhost/33663-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@50be1971 rejected from java.util.concurrent.ThreadPoolExecutor@1d92df[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:16,559 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:16,559 INFO [RS:2;jenkins-hbase4:35917] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,35917,1676016012831; zookeeper connection closed. 2023-02-10 08:00:16,560 ERROR [Listener at localhost/33663-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2956cdf7 rejected from java.util.concurrent.ThreadPoolExecutor@5cd4b8e2[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:16,560 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4740926] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4740926 2023-02-10 08:00:16,560 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): regionserver:35917-0x1003d3eafaf0003, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:16,560 ERROR [Listener at localhost/33663-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@516fabaf rejected from java.util.concurrent.ThreadPoolExecutor@5cd4b8e2[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:16,560 INFO [Listener at localhost/33663] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-10 08:00:16,660 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:16,660 INFO [M:0;jenkins-hbase4:45287] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,45287,1676016011592; zookeeper connection closed. 2023-02-10 08:00:16,660 ERROR [Listener at localhost/33663-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@437216db rejected from java.util.concurrent.ThreadPoolExecutor@32851166[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 24] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:16,660 DEBUG [Listener at localhost/33663-EventThread] zookeeper.ZKWatcher(600): master:45287-0x1003d3eafaf0000, quorum=127.0.0.1:60578, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:16,660 ERROR [Listener at localhost/33663-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@600201aa rejected from java.util.concurrent.ThreadPoolExecutor@32851166[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 24] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:16,662 WARN [Listener at localhost/33663] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-10 08:00:16,665 INFO [Listener at localhost/33663] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-10 08:00:16,767 WARN [BP-256912104-172.31.14.131-1676016008415 heartbeating to localhost/127.0.0.1:44445] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-10 08:00:16,767 WARN [BP-256912104-172.31.14.131-1676016008415 heartbeating to localhost/127.0.0.1:44445] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-256912104-172.31.14.131-1676016008415 (Datanode Uuid 25d7c9d1-863e-4964-bc32-77dcf2a8a658) service to localhost/127.0.0.1:44445 2023-02-10 08:00:16,769 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/cluster_489b6fd9-e39e-448e-c70d-be315a99e8c1/dfs/data/data5/current/BP-256912104-172.31.14.131-1676016008415] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 08:00:16,769 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/cluster_489b6fd9-e39e-448e-c70d-be315a99e8c1/dfs/data/data6/current/BP-256912104-172.31.14.131-1676016008415] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 08:00:16,770 WARN [Listener at localhost/33663] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-10 08:00:16,772 INFO [Listener at localhost/33663] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-10 08:00:16,874 WARN [BP-256912104-172.31.14.131-1676016008415 heartbeating to localhost/127.0.0.1:44445] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-10 08:00:16,874 WARN [BP-256912104-172.31.14.131-1676016008415 heartbeating to localhost/127.0.0.1:44445] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-256912104-172.31.14.131-1676016008415 (Datanode Uuid e08ec0d9-4f28-4dab-8a37-7cbe6cb2d5f1) service to localhost/127.0.0.1:44445 2023-02-10 08:00:16,875 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/cluster_489b6fd9-e39e-448e-c70d-be315a99e8c1/dfs/data/data3/current/BP-256912104-172.31.14.131-1676016008415] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 08:00:16,875 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/cluster_489b6fd9-e39e-448e-c70d-be315a99e8c1/dfs/data/data4/current/BP-256912104-172.31.14.131-1676016008415] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 08:00:16,875 WARN [Listener at localhost/33663] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-10 08:00:16,877 INFO [Listener at localhost/33663] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-10 08:00:16,979 WARN [BP-256912104-172.31.14.131-1676016008415 heartbeating to localhost/127.0.0.1:44445] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-10 08:00:16,979 WARN [BP-256912104-172.31.14.131-1676016008415 heartbeating to localhost/127.0.0.1:44445] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-256912104-172.31.14.131-1676016008415 (Datanode Uuid d25a848a-2cc2-4067-ad81-2ac90a87c200) service to localhost/127.0.0.1:44445 2023-02-10 08:00:16,980 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/cluster_489b6fd9-e39e-448e-c70d-be315a99e8c1/dfs/data/data1/current/BP-256912104-172.31.14.131-1676016008415] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 08:00:16,980 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/cluster_489b6fd9-e39e-448e-c70d-be315a99e8c1/dfs/data/data2/current/BP-256912104-172.31.14.131-1676016008415] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 08:00:17,004 INFO [Listener at localhost/33663] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-10 08:00:17,118 INFO [Listener at localhost/33663] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-10 08:00:17,156 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-02-10 08:00:17,168 INFO [Listener at localhost/33663] hbase.ResourceChecker(175): after: client.TestAsyncClusterAdminApi2#testStop Thread=77 (was 8) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager$NodeFailoverWorker.run(ReplicationSourceManager.java:703) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-5-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:44445 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: LeaseRenewer:jenkins@localhost:44445 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (267790143) connection to localhost/127.0.0.1:44445 from jenkins.hfs.2 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:44445 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (267790143) connection to localhost/127.0.0.1:44445 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (267790143) connection to localhost/127.0.0.1:44445 from jenkins.hfs.0 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager$NodeFailoverWorker.run(ReplicationSourceManager.java:703) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'NameNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (267790143) connection to localhost/127.0.0.1:44445 from jenkins.hfs.1 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:300) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-5-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/33663 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (267790143) connection to localhost/127.0.0.1:44445 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: ForkJoinPool-2-worker-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: HFileArchiver-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:44445 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@773b6a41 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-5-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=500 (was 260) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=288 (was 239) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 171), AvailableMemoryMB=3536 (was 4018) 2023-02-10 08:00:17,177 INFO [Listener at localhost/33663] hbase.ResourceChecker(147): before: client.TestAsyncClusterAdminApi2#testShutdown Thread=77, OpenFileDescriptor=500, MaxFileDescriptor=60000, SystemLoadAverage=288, ProcessCount=168, AvailableMemoryMB=3536 2023-02-10 08:00:17,178 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-10 08:00:17,178 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/hadoop.log.dir so I do NOT create it in target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9 2023-02-10 08:00:17,178 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f9e68d6d-1bf0-f356-31cf-b159d0f1b5c4/hadoop.tmp.dir so I do NOT create it in target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9 2023-02-10 08:00:17,178 INFO [Listener at localhost/33663] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/cluster_5d1ed0ef-257f-d997-688c-9c855b3f8051, deleteOnExit=true 2023-02-10 08:00:17,178 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-10 08:00:17,178 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/test.cache.data in system properties and HBase conf 2023-02-10 08:00:17,179 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/hadoop.tmp.dir in system properties and HBase conf 2023-02-10 08:00:17,179 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/hadoop.log.dir in system properties and HBase conf 2023-02-10 08:00:17,179 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-10 08:00:17,179 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-10 08:00:17,179 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-10 08:00:17,179 DEBUG [Listener at localhost/33663] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-10 08:00:17,180 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-10 08:00:17,180 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-10 08:00:17,180 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-10 08:00:17,180 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-10 08:00:17,180 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-10 08:00:17,180 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-10 08:00:17,181 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-10 08:00:17,181 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-10 08:00:17,181 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-10 08:00:17,181 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/nfs.dump.dir in system properties and HBase conf 2023-02-10 08:00:17,181 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/java.io.tmpdir in system properties and HBase conf 2023-02-10 08:00:17,181 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-10 08:00:17,181 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-10 08:00:17,181 INFO [Listener at localhost/33663] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-10 08:00:17,186 WARN [Listener at localhost/33663] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-10 08:00:17,186 WARN [Listener at localhost/33663] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-10 08:00:17,232 WARN [Listener at localhost/33663] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-10 08:00:17,234 INFO [Listener at localhost/33663] log.Slf4jLog(67): jetty-6.1.26 2023-02-10 08:00:17,239 INFO [Listener at localhost/33663] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/java.io.tmpdir/Jetty_localhost_45121_hdfs____.avqypz/webapp 2023-02-10 08:00:17,335 INFO [Listener at localhost/33663] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45121 2023-02-10 08:00:17,340 WARN [Listener at localhost/33663] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-10 08:00:17,341 WARN [Listener at localhost/33663] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-10 08:00:17,390 WARN [Listener at localhost/40139] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-10 08:00:17,401 WARN [Listener at localhost/40139] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-10 08:00:17,404 WARN [Listener at localhost/40139] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-10 08:00:17,405 INFO [Listener at localhost/40139] log.Slf4jLog(67): jetty-6.1.26 2023-02-10 08:00:17,410 INFO [Listener at localhost/40139] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/java.io.tmpdir/Jetty_localhost_45033_datanode____2bauom/webapp 2023-02-10 08:00:17,501 INFO [Listener at localhost/40139] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45033 2023-02-10 08:00:17,509 WARN [Listener at localhost/44009] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-10 08:00:17,532 WARN [Listener at localhost/44009] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-10 08:00:17,535 WARN [Listener at localhost/44009] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-10 08:00:17,536 INFO [Listener at localhost/44009] log.Slf4jLog(67): jetty-6.1.26 2023-02-10 08:00:17,547 INFO [Listener at localhost/44009] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/java.io.tmpdir/Jetty_localhost_38777_datanode____58ri61/webapp 2023-02-10 08:00:17,625 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x70c475b1c8de3669: Processing first storage report for DS-44341756-9f7c-4404-b9d2-0e41f34980d4 from datanode 54fe9603-238a-48ff-b4bf-c3f0a384c122 2023-02-10 08:00:17,626 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x70c475b1c8de3669: from storage DS-44341756-9f7c-4404-b9d2-0e41f34980d4 node DatanodeRegistration(127.0.0.1:45355, datanodeUuid=54fe9603-238a-48ff-b4bf-c3f0a384c122, infoPort=40719, infoSecurePort=0, ipcPort=44009, storageInfo=lv=-57;cid=testClusterID;nsid=47694378;c=1676016017189), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 08:00:17,626 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x70c475b1c8de3669: Processing first storage report for DS-8f5058cb-6692-4b11-9add-fbb30a917a9a from datanode 54fe9603-238a-48ff-b4bf-c3f0a384c122 2023-02-10 08:00:17,626 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x70c475b1c8de3669: from storage DS-8f5058cb-6692-4b11-9add-fbb30a917a9a node DatanodeRegistration(127.0.0.1:45355, datanodeUuid=54fe9603-238a-48ff-b4bf-c3f0a384c122, infoPort=40719, infoSecurePort=0, ipcPort=44009, storageInfo=lv=-57;cid=testClusterID;nsid=47694378;c=1676016017189), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 08:00:17,654 INFO [Listener at localhost/44009] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38777 2023-02-10 08:00:17,661 WARN [Listener at localhost/41845] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-10 08:00:17,685 WARN [Listener at localhost/41845] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-10 08:00:17,688 WARN [Listener at localhost/41845] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-10 08:00:17,690 INFO [Listener at localhost/41845] log.Slf4jLog(67): jetty-6.1.26 2023-02-10 08:00:17,696 INFO [Listener at localhost/41845] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/java.io.tmpdir/Jetty_localhost_34019_datanode____.o0kv9e/webapp 2023-02-10 08:00:17,768 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xdcca88162f25cbd3: Processing first storage report for DS-20c6f55b-ce02-45dd-aa08-e849112a1e2d from datanode 097a8492-c84c-44fa-91d0-5720bb534803 2023-02-10 08:00:17,768 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xdcca88162f25cbd3: from storage DS-20c6f55b-ce02-45dd-aa08-e849112a1e2d node DatanodeRegistration(127.0.0.1:33031, datanodeUuid=097a8492-c84c-44fa-91d0-5720bb534803, infoPort=35189, infoSecurePort=0, ipcPort=41845, storageInfo=lv=-57;cid=testClusterID;nsid=47694378;c=1676016017189), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 08:00:17,768 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xdcca88162f25cbd3: Processing first storage report for DS-5082a0f8-b184-436f-ab70-8d5b5d2cca2e from datanode 097a8492-c84c-44fa-91d0-5720bb534803 2023-02-10 08:00:17,768 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xdcca88162f25cbd3: from storage DS-5082a0f8-b184-436f-ab70-8d5b5d2cca2e node DatanodeRegistration(127.0.0.1:33031, datanodeUuid=097a8492-c84c-44fa-91d0-5720bb534803, infoPort=35189, infoSecurePort=0, ipcPort=41845, storageInfo=lv=-57;cid=testClusterID;nsid=47694378;c=1676016017189), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 08:00:17,802 INFO [Listener at localhost/41845] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34019 2023-02-10 08:00:17,809 WARN [Listener at localhost/34025] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-10 08:00:17,906 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x53ca07cf7128fbf6: Processing first storage report for DS-7e848d99-4ca0-4f20-8179-9cbe8a7f6c8d from datanode 980b38c3-e4f8-44c7-af10-c0b1aa20744e 2023-02-10 08:00:17,906 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x53ca07cf7128fbf6: from storage DS-7e848d99-4ca0-4f20-8179-9cbe8a7f6c8d node DatanodeRegistration(127.0.0.1:44351, datanodeUuid=980b38c3-e4f8-44c7-af10-c0b1aa20744e, infoPort=38307, infoSecurePort=0, ipcPort=34025, storageInfo=lv=-57;cid=testClusterID;nsid=47694378;c=1676016017189), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 08:00:17,906 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x53ca07cf7128fbf6: Processing first storage report for DS-874a5db4-d57f-4bcb-b98a-f8f1e6fa48b0 from datanode 980b38c3-e4f8-44c7-af10-c0b1aa20744e 2023-02-10 08:00:17,906 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x53ca07cf7128fbf6: from storage DS-874a5db4-d57f-4bcb-b98a-f8f1e6fa48b0 node DatanodeRegistration(127.0.0.1:44351, datanodeUuid=980b38c3-e4f8-44c7-af10-c0b1aa20744e, infoPort=38307, infoSecurePort=0, ipcPort=34025, storageInfo=lv=-57;cid=testClusterID;nsid=47694378;c=1676016017189), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 08:00:17,916 DEBUG [Listener at localhost/34025] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9 2023-02-10 08:00:17,921 INFO [Listener at localhost/34025] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/cluster_5d1ed0ef-257f-d997-688c-9c855b3f8051/zookeeper_0, clientPort=57601, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/cluster_5d1ed0ef-257f-d997-688c-9c855b3f8051/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/cluster_5d1ed0ef-257f-d997-688c-9c855b3f8051/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-10 08:00:17,922 INFO [Listener at localhost/34025] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=57601 2023-02-10 08:00:17,923 INFO [Listener at localhost/34025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:17,924 INFO [Listener at localhost/34025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:17,943 INFO [Listener at localhost/34025] util.FSUtils(479): Created version file at hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7 with version=8 2023-02-10 08:00:17,943 INFO [Listener at localhost/34025] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:44445/user/jenkins/test-data/513ea0f9-befc-76f7-a345-128d76168d12/hbase-staging 2023-02-10 08:00:17,945 INFO [Listener at localhost/34025] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=6 2023-02-10 08:00:17,945 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:17,946 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:17,946 INFO [Listener at localhost/34025] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-10 08:00:17,946 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:17,946 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-10 08:00:17,946 INFO [Listener at localhost/34025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-10 08:00:17,948 INFO [Listener at localhost/34025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35963 2023-02-10 08:00:17,948 INFO [Listener at localhost/34025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:17,950 INFO [Listener at localhost/34025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:17,951 INFO [Listener at localhost/34025] zookeeper.RecoverableZooKeeper(93): Process identifier=master:35963 connecting to ZooKeeper ensemble=127.0.0.1:57601 2023-02-10 08:00:17,960 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:359630x0, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-10 08:00:17,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:35963-0x1003d3ecbc50000 connected 2023-02-10 08:00:17,980 DEBUG [Listener at localhost/34025] zookeeper.ZKUtil(164): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 08:00:17,980 DEBUG [Listener at localhost/34025] zookeeper.ZKUtil(164): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 08:00:17,981 DEBUG [Listener at localhost/34025] zookeeper.ZKUtil(164): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-10 08:00:17,982 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35963 2023-02-10 08:00:17,983 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35963 2023-02-10 08:00:17,983 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35963 2023-02-10 08:00:17,983 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35963 2023-02-10 08:00:17,983 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35963 2023-02-10 08:00:17,984 INFO [Listener at localhost/34025] master.HMaster(439): hbase.rootdir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7, hbase.cluster.distributed=false 2023-02-10 08:00:18,001 INFO [Listener at localhost/34025] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-10 08:00:18,001 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:18,001 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:18,001 INFO [Listener at localhost/34025] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-10 08:00:18,001 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:18,001 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-10 08:00:18,001 INFO [Listener at localhost/34025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-10 08:00:18,003 INFO [Listener at localhost/34025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46575 2023-02-10 08:00:18,003 INFO [Listener at localhost/34025] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-10 08:00:18,004 DEBUG [Listener at localhost/34025] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-10 08:00:18,005 INFO [Listener at localhost/34025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:18,006 INFO [Listener at localhost/34025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:18,007 INFO [Listener at localhost/34025] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46575 connecting to ZooKeeper ensemble=127.0.0.1:57601 2023-02-10 08:00:18,011 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:465750x0, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-10 08:00:18,012 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:46575-0x1003d3ecbc50001 connected 2023-02-10 08:00:18,012 DEBUG [Listener at localhost/34025] zookeeper.ZKUtil(164): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 08:00:18,013 DEBUG [Listener at localhost/34025] zookeeper.ZKUtil(164): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 08:00:18,013 DEBUG [Listener at localhost/34025] zookeeper.ZKUtil(164): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-10 08:00:18,014 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46575 2023-02-10 08:00:18,016 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46575 2023-02-10 08:00:18,017 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46575 2023-02-10 08:00:18,019 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46575 2023-02-10 08:00:18,020 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46575 2023-02-10 08:00:18,031 INFO [Listener at localhost/34025] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-10 08:00:18,031 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:18,031 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:18,031 INFO [Listener at localhost/34025] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-10 08:00:18,031 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:18,031 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-10 08:00:18,031 INFO [Listener at localhost/34025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-10 08:00:18,033 INFO [Listener at localhost/34025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43847 2023-02-10 08:00:18,033 INFO [Listener at localhost/34025] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-10 08:00:18,036 DEBUG [Listener at localhost/34025] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-10 08:00:18,037 INFO [Listener at localhost/34025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:18,038 INFO [Listener at localhost/34025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:18,039 INFO [Listener at localhost/34025] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43847 connecting to ZooKeeper ensemble=127.0.0.1:57601 2023-02-10 08:00:18,046 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:438470x0, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-10 08:00:18,048 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:43847-0x1003d3ecbc50002 connected 2023-02-10 08:00:18,048 DEBUG [Listener at localhost/34025] zookeeper.ZKUtil(164): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 08:00:18,048 DEBUG [Listener at localhost/34025] zookeeper.ZKUtil(164): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 08:00:18,049 DEBUG [Listener at localhost/34025] zookeeper.ZKUtil(164): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-10 08:00:18,049 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43847 2023-02-10 08:00:18,049 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43847 2023-02-10 08:00:18,050 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43847 2023-02-10 08:00:18,052 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43847 2023-02-10 08:00:18,053 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43847 2023-02-10 08:00:18,065 INFO [Listener at localhost/34025] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-10 08:00:18,065 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:18,065 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:18,065 INFO [Listener at localhost/34025] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-10 08:00:18,065 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 08:00:18,065 INFO [Listener at localhost/34025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-10 08:00:18,065 INFO [Listener at localhost/34025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-10 08:00:18,067 INFO [Listener at localhost/34025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34017 2023-02-10 08:00:18,067 INFO [Listener at localhost/34025] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-10 08:00:18,068 DEBUG [Listener at localhost/34025] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-10 08:00:18,068 INFO [Listener at localhost/34025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:18,070 INFO [Listener at localhost/34025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:18,071 INFO [Listener at localhost/34025] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34017 connecting to ZooKeeper ensemble=127.0.0.1:57601 2023-02-10 08:00:18,074 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:340170x0, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-10 08:00:18,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:34017-0x1003d3ecbc50003 connected 2023-02-10 08:00:18,075 DEBUG [Listener at localhost/34025] zookeeper.ZKUtil(164): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 08:00:18,075 DEBUG [Listener at localhost/34025] zookeeper.ZKUtil(164): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 08:00:18,076 DEBUG [Listener at localhost/34025] zookeeper.ZKUtil(164): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-10 08:00:18,076 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34017 2023-02-10 08:00:18,076 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34017 2023-02-10 08:00:18,076 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34017 2023-02-10 08:00:18,077 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34017 2023-02-10 08:00:18,077 DEBUG [Listener at localhost/34025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34017 2023-02-10 08:00:18,078 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,35963,1676016017945 2023-02-10 08:00:18,080 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-10 08:00:18,080 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,35963,1676016017945 2023-02-10 08:00:18,083 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-10 08:00:18,083 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-10 08:00:18,083 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:18,083 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-10 08:00:18,084 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-10 08:00:18,085 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-10 08:00:18,086 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-10 08:00:18,087 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,35963,1676016017945 from backup master directory 2023-02-10 08:00:18,088 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,35963,1676016017945 2023-02-10 08:00:18,088 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-10 08:00:18,088 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-10 08:00:18,089 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,35963,1676016017945 2023-02-10 08:00:18,111 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/hbase.id with ID: ba8db3ee-ddfe-4d2b-a20e-07ac3d65c681 2023-02-10 08:00:18,126 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 08:00:18,129 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:18,143 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x50a4843f to 127.0.0.1:57601 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 08:00:18,148 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7be8309e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 08:00:18,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-10 08:00:18,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-10 08:00:18,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 08:00:18,151 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/MasterData/data/master/store-tmp 2023-02-10 08:00:18,165 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 08:00:18,165 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-10 08:00:18,165 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:18,165 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:18,165 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-10 08:00:18,166 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:18,166 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:18,166 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676016018165Disabling compacts and flushes for region at 1676016018165Disabling writes for close at 1676016018166 (+1 ms)Writing region close event to WAL at 1676016018166Closed at 1676016018166 2023-02-10 08:00:18,167 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/MasterData/WALs/jenkins-hbase4.apache.org,35963,1676016017945 2023-02-10 08:00:18,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35963%2C1676016017945, suffix=, logDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/MasterData/WALs/jenkins-hbase4.apache.org,35963,1676016017945, archiveDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/MasterData/oldWALs, maxLogs=10 2023-02-10 08:00:18,186 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33031,DS-20c6f55b-ce02-45dd-aa08-e849112a1e2d,DISK] 2023-02-10 08:00:18,187 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44351,DS-7e848d99-4ca0-4f20-8179-9cbe8a7f6c8d,DISK] 2023-02-10 08:00:18,187 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45355,DS-44341756-9f7c-4404-b9d2-0e41f34980d4,DISK] 2023-02-10 08:00:18,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/MasterData/WALs/jenkins-hbase4.apache.org,35963,1676016017945/jenkins-hbase4.apache.org%2C35963%2C1676016017945.1676016018170 2023-02-10 08:00:18,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33031,DS-20c6f55b-ce02-45dd-aa08-e849112a1e2d,DISK], DatanodeInfoWithStorage[127.0.0.1:45355,DS-44341756-9f7c-4404-b9d2-0e41f34980d4,DISK], DatanodeInfoWithStorage[127.0.0.1:44351,DS-7e848d99-4ca0-4f20-8179-9cbe8a7f6c8d,DISK]] 2023-02-10 08:00:18,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-10 08:00:18,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 08:00:18,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-10 08:00:18,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-10 08:00:18,195 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-10 08:00:18,197 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-10 08:00:18,198 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-10 08:00:18,198 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:18,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-10 08:00:18,200 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-10 08:00:18,203 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-10 08:00:18,206 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 08:00:18,206 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74494707, jitterRate=0.11005763709545135}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-10 08:00:18,206 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676016018192Initializing all the Stores at 1676016018194 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676016018194Cleaning up temporary data from old regions at 1676016018201 (+7 ms)Cleaning up detritus from prior splits at 1676016018201Region opened successfully at 1676016018206 (+5 ms) 2023-02-10 08:00:18,207 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-10 08:00:18,208 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-10 08:00:18,208 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-10 08:00:18,208 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 08:00:18,208 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 08:00:18,209 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-02-10 08:00:18,209 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-10 08:00:18,210 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-10 08:00:18,211 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-10 08:00:18,221 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-10 08:00:18,221 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-10 08:00:18,222 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-10 08:00:18,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-10 08:00:18,222 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-10 08:00:18,225 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:18,226 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-10 08:00:18,226 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-10 08:00:18,227 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-10 08:00:18,229 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-10 08:00:18,229 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-10 08:00:18,229 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:18,229 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-10 08:00:18,229 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-10 08:00:18,229 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,35963,1676016017945, sessionid=0x1003d3ecbc50000, setting cluster-up flag (Was=false) 2023-02-10 08:00:18,234 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:18,239 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-10 08:00:18,240 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,35963,1676016017945 2023-02-10 08:00:18,244 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:18,249 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-10 08:00:18,250 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,35963,1676016017945 2023-02-10 08:00:18,250 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/.hbase-snapshot/.tmp 2023-02-10 08:00:18,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-10 08:00:18,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-10 08:00:18,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-10 08:00:18,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-10 08:00:18,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-10 08:00:18,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-10 08:00:18,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-10 08:00:18,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,255 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676016048255 2023-02-10 08:00:18,255 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-10 08:00:18,255 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-10 08:00:18,256 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-10 08:00:18,256 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-10 08:00:18,256 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-10 08:00:18,256 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-10 08:00:18,256 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,256 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-10 08:00:18,257 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-10 08:00:18,257 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-10 08:00:18,257 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-10 08:00:18,257 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-10 08:00:18,257 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-10 08:00:18,257 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-10 08:00:18,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676016018257,5,FailOnTimeoutGroup] 2023-02-10 08:00:18,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676016018257,5,FailOnTimeoutGroup] 2023-02-10 08:00:18,258 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,258 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-10 08:00:18,258 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,258 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,259 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-10 08:00:18,277 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-10 08:00:18,279 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-10 08:00:18,279 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7 2023-02-10 08:00:18,279 INFO [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(952): ClusterId : ba8db3ee-ddfe-4d2b-a20e-07ac3d65c681 2023-02-10 08:00:18,279 INFO [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(952): ClusterId : ba8db3ee-ddfe-4d2b-a20e-07ac3d65c681 2023-02-10 08:00:18,280 DEBUG [RS:0;jenkins-hbase4:46575] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-10 08:00:18,281 DEBUG [RS:2;jenkins-hbase4:34017] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-10 08:00:18,279 INFO [RS:1;jenkins-hbase4:43847] regionserver.HRegionServer(952): ClusterId : ba8db3ee-ddfe-4d2b-a20e-07ac3d65c681 2023-02-10 08:00:18,282 DEBUG [RS:1;jenkins-hbase4:43847] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-10 08:00:18,287 DEBUG [RS:0;jenkins-hbase4:46575] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-10 08:00:18,287 DEBUG [RS:2;jenkins-hbase4:34017] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-10 08:00:18,287 DEBUG [RS:0;jenkins-hbase4:46575] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-10 08:00:18,287 DEBUG [RS:2;jenkins-hbase4:34017] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-10 08:00:18,287 DEBUG [RS:1;jenkins-hbase4:43847] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-10 08:00:18,288 DEBUG [RS:1;jenkins-hbase4:43847] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-10 08:00:18,291 DEBUG [RS:0;jenkins-hbase4:46575] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-10 08:00:18,293 DEBUG [RS:2;jenkins-hbase4:34017] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-10 08:00:18,294 DEBUG [RS:2;jenkins-hbase4:34017] zookeeper.ReadOnlyZKClient(139): Connect 0x74756ad6 to 127.0.0.1:57601 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 08:00:18,294 DEBUG [RS:0;jenkins-hbase4:46575] zookeeper.ReadOnlyZKClient(139): Connect 0x425b4345 to 127.0.0.1:57601 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 08:00:18,295 DEBUG [RS:1;jenkins-hbase4:43847] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-10 08:00:18,296 DEBUG [RS:1;jenkins-hbase4:43847] zookeeper.ReadOnlyZKClient(139): Connect 0x4adee753 to 127.0.0.1:57601 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 08:00:18,301 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 08:00:18,303 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-10 08:00:18,304 DEBUG [RS:1;jenkins-hbase4:43847] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@44190078, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 08:00:18,304 DEBUG [RS:2;jenkins-hbase4:34017] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@704af080, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 08:00:18,304 DEBUG [RS:1;jenkins-hbase4:43847] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@40268c23, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-10 08:00:18,304 DEBUG [RS:2;jenkins-hbase4:34017] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@354860ed, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-10 08:00:18,305 DEBUG [RS:0;jenkins-hbase4:46575] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@366497e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 08:00:18,306 DEBUG [RS:0;jenkins-hbase4:46575] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2af6e430, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-10 08:00:18,309 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/info 2023-02-10 08:00:18,310 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-10 08:00:18,311 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:18,311 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-10 08:00:18,313 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/rep_barrier 2023-02-10 08:00:18,314 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-10 08:00:18,314 DEBUG [RS:2;jenkins-hbase4:34017] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:34017 2023-02-10 08:00:18,315 INFO [RS:2;jenkins-hbase4:34017] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-10 08:00:18,315 INFO [RS:2;jenkins-hbase4:34017] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-10 08:00:18,315 DEBUG [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(1023): About to register with Master. 2023-02-10 08:00:18,315 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:18,316 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-10 08:00:18,316 DEBUG [RS:0;jenkins-hbase4:46575] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:46575 2023-02-10 08:00:18,316 INFO [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,35963,1676016017945 with isa=jenkins-hbase4.apache.org/172.31.14.131:34017, startcode=1676016018064 2023-02-10 08:00:18,316 INFO [RS:0;jenkins-hbase4:46575] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-10 08:00:18,316 INFO [RS:0;jenkins-hbase4:46575] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-10 08:00:18,316 DEBUG [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(1023): About to register with Master. 2023-02-10 08:00:18,316 DEBUG [RS:2;jenkins-hbase4:34017] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-10 08:00:18,317 DEBUG [RS:1;jenkins-hbase4:43847] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:43847 2023-02-10 08:00:18,317 INFO [RS:1;jenkins-hbase4:43847] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-10 08:00:18,317 INFO [RS:1;jenkins-hbase4:43847] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-10 08:00:18,317 DEBUG [RS:1;jenkins-hbase4:43847] regionserver.HRegionServer(1023): About to register with Master. 2023-02-10 08:00:18,317 INFO [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,35963,1676016017945 with isa=jenkins-hbase4.apache.org/172.31.14.131:46575, startcode=1676016018000 2023-02-10 08:00:18,317 DEBUG [RS:0;jenkins-hbase4:46575] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-10 08:00:18,318 INFO [RS:1;jenkins-hbase4:43847] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,35963,1676016017945 with isa=jenkins-hbase4.apache.org/172.31.14.131:43847, startcode=1676016018030 2023-02-10 08:00:18,318 DEBUG [RS:1;jenkins-hbase4:43847] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-10 08:00:18,318 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/table 2023-02-10 08:00:18,320 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-10 08:00:18,321 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38167, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-02-10 08:00:18,321 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49859, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-02-10 08:00:18,321 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37453, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-02-10 08:00:18,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35963] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:18,323 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35963] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:18,321 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:18,323 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35963] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43847,1676016018030 2023-02-10 08:00:18,324 DEBUG [RS:1;jenkins-hbase4:43847] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7 2023-02-10 08:00:18,324 DEBUG [RS:1;jenkins-hbase4:43847] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:40139 2023-02-10 08:00:18,325 DEBUG [RS:1;jenkins-hbase4:43847] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-10 08:00:18,325 DEBUG [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7 2023-02-10 08:00:18,325 DEBUG [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:40139 2023-02-10 08:00:18,325 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740 2023-02-10 08:00:18,325 DEBUG [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-10 08:00:18,325 DEBUG [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7 2023-02-10 08:00:18,325 DEBUG [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:40139 2023-02-10 08:00:18,325 DEBUG [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-10 08:00:18,326 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740 2023-02-10 08:00:18,327 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:18,330 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-10 08:00:18,331 DEBUG [RS:1;jenkins-hbase4:43847] zookeeper.ZKUtil(162): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43847,1676016018030 2023-02-10 08:00:18,331 WARN [RS:1;jenkins-hbase4:43847] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-10 08:00:18,331 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43847,1676016018030] 2023-02-10 08:00:18,331 INFO [RS:1;jenkins-hbase4:43847] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 08:00:18,331 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46575,1676016018000] 2023-02-10 08:00:18,331 DEBUG [RS:1;jenkins-hbase4:43847] regionserver.HRegionServer(1947): logDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/WALs/jenkins-hbase4.apache.org,43847,1676016018030 2023-02-10 08:00:18,331 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34017,1676016018064] 2023-02-10 08:00:18,332 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-10 08:00:18,332 DEBUG [RS:0;jenkins-hbase4:46575] zookeeper.ZKUtil(162): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:18,332 DEBUG [RS:2;jenkins-hbase4:34017] zookeeper.ZKUtil(162): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:18,333 WARN [RS:0;jenkins-hbase4:46575] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-10 08:00:18,333 WARN [RS:2;jenkins-hbase4:34017] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-10 08:00:18,334 INFO [RS:0;jenkins-hbase4:46575] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 08:00:18,334 INFO [RS:2;jenkins-hbase4:34017] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 08:00:18,334 DEBUG [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(1947): logDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/WALs/jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:18,334 DEBUG [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(1947): logDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/WALs/jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:18,339 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 08:00:18,339 DEBUG [RS:1;jenkins-hbase4:43847] zookeeper.ZKUtil(162): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43847,1676016018030 2023-02-10 08:00:18,340 DEBUG [RS:1;jenkins-hbase4:43847] zookeeper.ZKUtil(162): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:18,340 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75284844, jitterRate=0.12183159589767456}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-10 08:00:18,340 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676016018301Initializing all the Stores at 1676016018303 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676016018303Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676016018303Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676016018303Cleaning up temporary data from old regions at 1676016018328 (+25 ms)Cleaning up detritus from prior splits at 1676016018328Region opened successfully at 1676016018340 (+12 ms) 2023-02-10 08:00:18,340 DEBUG [RS:1;jenkins-hbase4:43847] zookeeper.ZKUtil(162): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:18,341 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-10 08:00:18,341 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-10 08:00:18,341 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-10 08:00:18,341 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-10 08:00:18,341 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-10 08:00:18,341 DEBUG [RS:0;jenkins-hbase4:46575] zookeeper.ZKUtil(162): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43847,1676016018030 2023-02-10 08:00:18,342 DEBUG [RS:0;jenkins-hbase4:46575] zookeeper.ZKUtil(162): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:18,342 DEBUG [RS:1;jenkins-hbase4:43847] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-10 08:00:18,342 INFO [RS:1;jenkins-hbase4:43847] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-10 08:00:18,344 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-10 08:00:18,344 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676016018341Disabling compacts and flushes for region at 1676016018341Disabling writes for close at 1676016018341Writing region close event to WAL at 1676016018344 (+3 ms)Closed at 1676016018344 2023-02-10 08:00:18,344 DEBUG [RS:0;jenkins-hbase4:46575] zookeeper.ZKUtil(162): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:18,345 INFO [RS:1;jenkins-hbase4:43847] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-10 08:00:18,346 INFO [RS:1;jenkins-hbase4:43847] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-10 08:00:18,346 INFO [RS:1;jenkins-hbase4:43847] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,346 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-10 08:00:18,346 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-10 08:00:18,346 INFO [RS:1;jenkins-hbase4:43847] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-10 08:00:18,346 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-10 08:00:18,348 INFO [RS:1;jenkins-hbase4:43847] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,348 DEBUG [RS:1;jenkins-hbase4:43847] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,349 DEBUG [RS:1;jenkins-hbase4:43847] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,349 DEBUG [RS:2;jenkins-hbase4:34017] zookeeper.ZKUtil(162): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43847,1676016018030 2023-02-10 08:00:18,349 DEBUG [RS:0;jenkins-hbase4:46575] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-10 08:00:18,349 DEBUG [RS:1;jenkins-hbase4:43847] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,349 INFO [RS:0;jenkins-hbase4:46575] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-10 08:00:18,350 DEBUG [RS:2;jenkins-hbase4:34017] zookeeper.ZKUtil(162): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:18,351 DEBUG [RS:2;jenkins-hbase4:34017] zookeeper.ZKUtil(162): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:18,349 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-10 08:00:18,349 DEBUG [RS:1;jenkins-hbase4:43847] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,352 DEBUG [RS:1;jenkins-hbase4:43847] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,356 DEBUG [RS:1;jenkins-hbase4:43847] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-10 08:00:18,356 INFO [RS:0;jenkins-hbase4:46575] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-10 08:00:18,356 DEBUG [RS:1;jenkins-hbase4:43847] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,357 DEBUG [RS:2;jenkins-hbase4:34017] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-10 08:00:18,357 DEBUG [RS:1;jenkins-hbase4:43847] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,357 INFO [RS:2;jenkins-hbase4:34017] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-10 08:00:18,357 DEBUG [RS:1;jenkins-hbase4:43847] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,357 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-10 08:00:18,357 INFO [RS:0;jenkins-hbase4:46575] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-10 08:00:18,357 DEBUG [RS:1;jenkins-hbase4:43847] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,357 INFO [RS:0;jenkins-hbase4:46575] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,360 INFO [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-10 08:00:18,361 INFO [RS:2;jenkins-hbase4:34017] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-10 08:00:18,364 INFO [RS:1;jenkins-hbase4:43847] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,364 INFO [RS:1;jenkins-hbase4:43847] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,365 INFO [RS:0;jenkins-hbase4:46575] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,365 INFO [RS:2;jenkins-hbase4:34017] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-10 08:00:18,365 INFO [RS:1;jenkins-hbase4:43847] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,365 DEBUG [RS:0;jenkins-hbase4:46575] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,365 INFO [RS:2;jenkins-hbase4:34017] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,366 DEBUG [RS:0;jenkins-hbase4:46575] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,366 DEBUG [RS:0;jenkins-hbase4:46575] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,366 DEBUG [RS:0;jenkins-hbase4:46575] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,366 DEBUG [RS:0;jenkins-hbase4:46575] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,366 DEBUG [RS:0;jenkins-hbase4:46575] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-10 08:00:18,366 DEBUG [RS:0;jenkins-hbase4:46575] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,366 DEBUG [RS:0;jenkins-hbase4:46575] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,366 DEBUG [RS:0;jenkins-hbase4:46575] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,366 DEBUG [RS:0;jenkins-hbase4:46575] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,368 INFO [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-10 08:00:18,372 INFO [RS:0;jenkins-hbase4:46575] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,372 INFO [RS:0;jenkins-hbase4:46575] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,372 INFO [RS:0;jenkins-hbase4:46575] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,378 INFO [RS:2;jenkins-hbase4:34017] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,378 DEBUG [RS:2;jenkins-hbase4:34017] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,378 DEBUG [RS:2;jenkins-hbase4:34017] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,378 DEBUG [RS:2;jenkins-hbase4:34017] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,378 DEBUG [RS:2;jenkins-hbase4:34017] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,378 DEBUG [RS:2;jenkins-hbase4:34017] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,378 DEBUG [RS:2;jenkins-hbase4:34017] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-10 08:00:18,379 DEBUG [RS:2;jenkins-hbase4:34017] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,379 DEBUG [RS:2;jenkins-hbase4:34017] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,379 DEBUG [RS:2;jenkins-hbase4:34017] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,379 DEBUG [RS:2;jenkins-hbase4:34017] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 08:00:18,380 INFO [RS:2;jenkins-hbase4:34017] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,380 INFO [RS:2;jenkins-hbase4:34017] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,380 INFO [RS:2;jenkins-hbase4:34017] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,382 INFO [RS:1;jenkins-hbase4:43847] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-10 08:00:18,382 INFO [RS:1;jenkins-hbase4:43847] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43847,1676016018030-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,384 INFO [RS:0;jenkins-hbase4:46575] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-10 08:00:18,384 INFO [RS:0;jenkins-hbase4:46575] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46575,1676016018000-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,391 INFO [RS:2;jenkins-hbase4:34017] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-10 08:00:18,391 INFO [RS:2;jenkins-hbase4:34017] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34017,1676016018064-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,397 INFO [RS:1;jenkins-hbase4:43847] regionserver.Replication(203): jenkins-hbase4.apache.org,43847,1676016018030 started 2023-02-10 08:00:18,397 INFO [RS:1;jenkins-hbase4:43847] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,43847,1676016018030, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43847, sessionid=0x1003d3ecbc50002 2023-02-10 08:00:18,397 DEBUG [RS:1;jenkins-hbase4:43847] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-10 08:00:18,397 DEBUG [RS:1;jenkins-hbase4:43847] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43847,1676016018030 2023-02-10 08:00:18,397 DEBUG [RS:1;jenkins-hbase4:43847] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43847,1676016018030' 2023-02-10 08:00:18,397 DEBUG [RS:1;jenkins-hbase4:43847] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 08:00:18,398 DEBUG [RS:1;jenkins-hbase4:43847] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 08:00:18,398 DEBUG [RS:1;jenkins-hbase4:43847] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-10 08:00:18,398 DEBUG [RS:1;jenkins-hbase4:43847] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-10 08:00:18,398 DEBUG [RS:1;jenkins-hbase4:43847] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43847,1676016018030 2023-02-10 08:00:18,398 DEBUG [RS:1;jenkins-hbase4:43847] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43847,1676016018030' 2023-02-10 08:00:18,398 DEBUG [RS:1;jenkins-hbase4:43847] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-10 08:00:18,399 DEBUG [RS:1;jenkins-hbase4:43847] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-10 08:00:18,400 DEBUG [RS:1;jenkins-hbase4:43847] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-10 08:00:18,400 INFO [RS:1;jenkins-hbase4:43847] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-10 08:00:18,400 INFO [RS:1;jenkins-hbase4:43847] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-10 08:00:18,402 INFO [RS:0;jenkins-hbase4:46575] regionserver.Replication(203): jenkins-hbase4.apache.org,46575,1676016018000 started 2023-02-10 08:00:18,402 INFO [RS:2;jenkins-hbase4:34017] regionserver.Replication(203): jenkins-hbase4.apache.org,34017,1676016018064 started 2023-02-10 08:00:18,402 INFO [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,46575,1676016018000, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46575, sessionid=0x1003d3ecbc50001 2023-02-10 08:00:18,402 INFO [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,34017,1676016018064, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34017, sessionid=0x1003d3ecbc50003 2023-02-10 08:00:18,402 DEBUG [RS:0;jenkins-hbase4:46575] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-10 08:00:18,402 DEBUG [RS:0;jenkins-hbase4:46575] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:18,403 DEBUG [RS:0;jenkins-hbase4:46575] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46575,1676016018000' 2023-02-10 08:00:18,402 DEBUG [RS:2;jenkins-hbase4:34017] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-10 08:00:18,403 DEBUG [RS:2;jenkins-hbase4:34017] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:18,403 DEBUG [RS:2;jenkins-hbase4:34017] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34017,1676016018064' 2023-02-10 08:00:18,403 DEBUG [RS:0;jenkins-hbase4:46575] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 08:00:18,403 DEBUG [RS:2;jenkins-hbase4:34017] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 08:00:18,404 DEBUG [RS:0;jenkins-hbase4:46575] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 08:00:18,404 DEBUG [RS:2;jenkins-hbase4:34017] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 08:00:18,404 DEBUG [RS:0;jenkins-hbase4:46575] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-10 08:00:18,404 DEBUG [RS:0;jenkins-hbase4:46575] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-10 08:00:18,404 DEBUG [RS:2;jenkins-hbase4:34017] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-10 08:00:18,405 DEBUG [RS:2;jenkins-hbase4:34017] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-10 08:00:18,404 DEBUG [RS:0;jenkins-hbase4:46575] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:18,405 DEBUG [RS:2;jenkins-hbase4:34017] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:18,405 DEBUG [RS:0;jenkins-hbase4:46575] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46575,1676016018000' 2023-02-10 08:00:18,405 DEBUG [RS:0;jenkins-hbase4:46575] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-10 08:00:18,405 DEBUG [RS:2;jenkins-hbase4:34017] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34017,1676016018064' 2023-02-10 08:00:18,405 DEBUG [RS:2;jenkins-hbase4:34017] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-10 08:00:18,405 DEBUG [RS:0;jenkins-hbase4:46575] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-10 08:00:18,405 DEBUG [RS:2;jenkins-hbase4:34017] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-10 08:00:18,405 DEBUG [RS:0;jenkins-hbase4:46575] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-10 08:00:18,406 INFO [RS:0;jenkins-hbase4:46575] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-10 08:00:18,406 INFO [RS:0;jenkins-hbase4:46575] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-10 08:00:18,406 DEBUG [RS:2;jenkins-hbase4:34017] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-10 08:00:18,406 INFO [RS:2;jenkins-hbase4:34017] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-10 08:00:18,406 INFO [RS:2;jenkins-hbase4:34017] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-10 08:00:18,503 INFO [RS:1;jenkins-hbase4:43847] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43847%2C1676016018030, suffix=, logDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/WALs/jenkins-hbase4.apache.org,43847,1676016018030, archiveDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/oldWALs, maxLogs=32 2023-02-10 08:00:18,508 DEBUG [jenkins-hbase4:35963] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-10 08:00:18,508 DEBUG [jenkins-hbase4:35963] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 08:00:18,509 INFO [RS:0;jenkins-hbase4:46575] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46575%2C1676016018000, suffix=, logDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/WALs/jenkins-hbase4.apache.org,46575,1676016018000, archiveDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/oldWALs, maxLogs=32 2023-02-10 08:00:18,509 INFO [RS:2;jenkins-hbase4:34017] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34017%2C1676016018064, suffix=, logDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/WALs/jenkins-hbase4.apache.org,34017,1676016018064, archiveDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/oldWALs, maxLogs=32 2023-02-10 08:00:18,510 DEBUG [jenkins-hbase4:35963] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 08:00:18,510 DEBUG [jenkins-hbase4:35963] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 08:00:18,510 DEBUG [jenkins-hbase4:35963] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 08:00:18,510 DEBUG [jenkins-hbase4:35963] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 08:00:18,511 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34017,1676016018064, state=OPENING 2023-02-10 08:00:18,513 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-10 08:00:18,514 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:18,515 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-10 08:00:18,515 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34017,1676016018064}] 2023-02-10 08:00:18,531 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45355,DS-44341756-9f7c-4404-b9d2-0e41f34980d4,DISK] 2023-02-10 08:00:18,532 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44351,DS-7e848d99-4ca0-4f20-8179-9cbe8a7f6c8d,DISK] 2023-02-10 08:00:18,532 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33031,DS-20c6f55b-ce02-45dd-aa08-e849112a1e2d,DISK] 2023-02-10 08:00:18,535 INFO [RS:1;jenkins-hbase4:43847] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/WALs/jenkins-hbase4.apache.org,43847,1676016018030/jenkins-hbase4.apache.org%2C43847%2C1676016018030.1676016018504 2023-02-10 08:00:18,536 DEBUG [RS:1;jenkins-hbase4:43847] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45355,DS-44341756-9f7c-4404-b9d2-0e41f34980d4,DISK], DatanodeInfoWithStorage[127.0.0.1:44351,DS-7e848d99-4ca0-4f20-8179-9cbe8a7f6c8d,DISK], DatanodeInfoWithStorage[127.0.0.1:33031,DS-20c6f55b-ce02-45dd-aa08-e849112a1e2d,DISK]] 2023-02-10 08:00:18,547 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44351,DS-7e848d99-4ca0-4f20-8179-9cbe8a7f6c8d,DISK] 2023-02-10 08:00:18,548 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33031,DS-20c6f55b-ce02-45dd-aa08-e849112a1e2d,DISK] 2023-02-10 08:00:18,548 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45355,DS-44341756-9f7c-4404-b9d2-0e41f34980d4,DISK] 2023-02-10 08:00:18,549 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33031,DS-20c6f55b-ce02-45dd-aa08-e849112a1e2d,DISK] 2023-02-10 08:00:18,549 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45355,DS-44341756-9f7c-4404-b9d2-0e41f34980d4,DISK] 2023-02-10 08:00:18,558 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44351,DS-7e848d99-4ca0-4f20-8179-9cbe8a7f6c8d,DISK] 2023-02-10 08:00:18,560 INFO [RS:2;jenkins-hbase4:34017] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/WALs/jenkins-hbase4.apache.org,34017,1676016018064/jenkins-hbase4.apache.org%2C34017%2C1676016018064.1676016018513 2023-02-10 08:00:18,560 DEBUG [RS:2;jenkins-hbase4:34017] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45355,DS-44341756-9f7c-4404-b9d2-0e41f34980d4,DISK], DatanodeInfoWithStorage[127.0.0.1:33031,DS-20c6f55b-ce02-45dd-aa08-e849112a1e2d,DISK], DatanodeInfoWithStorage[127.0.0.1:44351,DS-7e848d99-4ca0-4f20-8179-9cbe8a7f6c8d,DISK]] 2023-02-10 08:00:18,560 INFO [RS:0;jenkins-hbase4:46575] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/WALs/jenkins-hbase4.apache.org,46575,1676016018000/jenkins-hbase4.apache.org%2C46575%2C1676016018000.1676016018513 2023-02-10 08:00:18,561 DEBUG [RS:0;jenkins-hbase4:46575] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45355,DS-44341756-9f7c-4404-b9d2-0e41f34980d4,DISK], DatanodeInfoWithStorage[127.0.0.1:33031,DS-20c6f55b-ce02-45dd-aa08-e849112a1e2d,DISK], DatanodeInfoWithStorage[127.0.0.1:44351,DS-7e848d99-4ca0-4f20-8179-9cbe8a7f6c8d,DISK]] 2023-02-10 08:00:18,670 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:18,671 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-10 08:00:18,672 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51362, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-10 08:00:18,677 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-10 08:00:18,677 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 08:00:18,679 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34017%2C1676016018064.meta, suffix=.meta, logDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/WALs/jenkins-hbase4.apache.org,34017,1676016018064, archiveDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/oldWALs, maxLogs=32 2023-02-10 08:00:18,695 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33031,DS-20c6f55b-ce02-45dd-aa08-e849112a1e2d,DISK] 2023-02-10 08:00:18,696 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44351,DS-7e848d99-4ca0-4f20-8179-9cbe8a7f6c8d,DISK] 2023-02-10 08:00:18,695 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45355,DS-44341756-9f7c-4404-b9d2-0e41f34980d4,DISK] 2023-02-10 08:00:18,700 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/WALs/jenkins-hbase4.apache.org,34017,1676016018064/jenkins-hbase4.apache.org%2C34017%2C1676016018064.meta.1676016018680.meta 2023-02-10 08:00:18,700 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44351,DS-7e848d99-4ca0-4f20-8179-9cbe8a7f6c8d,DISK], DatanodeInfoWithStorage[127.0.0.1:33031,DS-20c6f55b-ce02-45dd-aa08-e849112a1e2d,DISK], DatanodeInfoWithStorage[127.0.0.1:45355,DS-44341756-9f7c-4404-b9d2-0e41f34980d4,DISK]] 2023-02-10 08:00:18,701 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-10 08:00:18,701 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-10 08:00:18,701 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-10 08:00:18,704 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-10 08:00:18,704 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-10 08:00:18,704 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 08:00:18,704 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-10 08:00:18,704 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-10 08:00:18,706 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-10 08:00:18,708 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/info 2023-02-10 08:00:18,708 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/info 2023-02-10 08:00:18,708 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-10 08:00:18,709 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:18,709 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-10 08:00:18,710 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/rep_barrier 2023-02-10 08:00:18,711 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/rep_barrier 2023-02-10 08:00:18,711 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-10 08:00:18,712 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:18,712 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-10 08:00:18,713 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/table 2023-02-10 08:00:18,713 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/table 2023-02-10 08:00:18,713 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-10 08:00:18,714 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:18,715 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740 2023-02-10 08:00:18,717 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740 2023-02-10 08:00:18,721 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-10 08:00:18,722 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-10 08:00:18,723 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70216088, jitterRate=0.04630124568939209}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-10 08:00:18,724 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676016018704Writing region info on filesystem at 1676016018704Initializing all the Stores at 1676016018706 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676016018706Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676016018706Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676016018706Cleaning up temporary data from old regions at 1676016018718 (+12 ms)Cleaning up detritus from prior splits at 1676016018720 (+2 ms)Running coprocessor post-open hooks at 1676016018723 (+3 ms)Region opened successfully at 1676016018723 2023-02-10 08:00:18,725 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676016018670 2023-02-10 08:00:18,730 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-10 08:00:18,730 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-10 08:00:18,731 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34017,1676016018064, state=OPEN 2023-02-10 08:00:18,733 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-10 08:00:18,733 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-10 08:00:18,736 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-10 08:00:18,736 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34017,1676016018064 in 218 msec 2023-02-10 08:00:18,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-10 08:00:18,740 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 390 msec 2023-02-10 08:00:18,743 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 489 msec 2023-02-10 08:00:18,743 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676016018081, completionTime=-1 2023-02-10 08:00:18,743 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-10 08:00:18,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 08:00:18,747 DEBUG [hconnection-0x466f1647-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-10 08:00:18,749 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51366, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-10 08:00:18,750 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 08:00:18,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676016078750 2023-02-10 08:00:18,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676016138751 2023-02-10 08:00:18,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 7 msec 2023-02-10 08:00:18,758 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35963,1676016017945-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35963,1676016017945-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35963,1676016017945-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:35963, period=300000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-10 08:00:18,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-10 08:00:18,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-10 08:00:18,760 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-10 08:00:18,760 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-10 08:00:18,762 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-10 08:00:18,764 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-10 08:00:18,766 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/.tmp/data/hbase/namespace/9fdf2ee53ef46eb3663ff4b0d9bf4562 2023-02-10 08:00:18,767 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/.tmp/data/hbase/namespace/9fdf2ee53ef46eb3663ff4b0d9bf4562 empty. 2023-02-10 08:00:18,767 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/.tmp/data/hbase/namespace/9fdf2ee53ef46eb3663ff4b0d9bf4562 2023-02-10 08:00:18,767 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-10 08:00:18,785 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-10 08:00:18,787 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 9fdf2ee53ef46eb3663ff4b0d9bf4562, NAME => 'hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/.tmp 2023-02-10 08:00:18,803 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 08:00:18,803 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 9fdf2ee53ef46eb3663ff4b0d9bf4562, disabling compactions & flushes 2023-02-10 08:00:18,803 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. 2023-02-10 08:00:18,803 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. 2023-02-10 08:00:18,803 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. after waiting 0 ms 2023-02-10 08:00:18,803 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. 2023-02-10 08:00:18,803 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. 2023-02-10 08:00:18,803 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 9fdf2ee53ef46eb3663ff4b0d9bf4562: Waiting for close lock at 1676016018803Disabling compacts and flushes for region at 1676016018803Disabling writes for close at 1676016018803Writing region close event to WAL at 1676016018803Closed at 1676016018803 2023-02-10 08:00:18,806 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-10 08:00:18,808 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676016018808"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676016018808"}]},"ts":"1676016018808"} 2023-02-10 08:00:18,811 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-10 08:00:18,812 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-10 08:00:18,812 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676016018812"}]},"ts":"1676016018812"} 2023-02-10 08:00:18,814 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-10 08:00:18,818 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 08:00:18,819 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 08:00:18,819 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 08:00:18,819 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 08:00:18,819 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 08:00:18,819 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=9fdf2ee53ef46eb3663ff4b0d9bf4562, ASSIGN}] 2023-02-10 08:00:18,821 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=9fdf2ee53ef46eb3663ff4b0d9bf4562, ASSIGN 2023-02-10 08:00:18,822 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=9fdf2ee53ef46eb3663ff4b0d9bf4562, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46575,1676016018000; forceNewPlan=false, retain=false 2023-02-10 08:00:18,973 INFO [jenkins-hbase4:35963] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 08:00:18,974 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=9fdf2ee53ef46eb3663ff4b0d9bf4562, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:18,974 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676016018974"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676016018974"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676016018974"}]},"ts":"1676016018974"} 2023-02-10 08:00:18,977 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 9fdf2ee53ef46eb3663ff4b0d9bf4562, server=jenkins-hbase4.apache.org,46575,1676016018000}] 2023-02-10 08:00:19,131 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:19,131 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-10 08:00:19,133 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44160, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-10 08:00:19,138 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. 2023-02-10 08:00:19,138 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9fdf2ee53ef46eb3663ff4b0d9bf4562, NAME => 'hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562.', STARTKEY => '', ENDKEY => ''} 2023-02-10 08:00:19,138 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 9fdf2ee53ef46eb3663ff4b0d9bf4562 2023-02-10 08:00:19,138 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 08:00:19,138 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9fdf2ee53ef46eb3663ff4b0d9bf4562 2023-02-10 08:00:19,138 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9fdf2ee53ef46eb3663ff4b0d9bf4562 2023-02-10 08:00:19,140 INFO [StoreOpener-9fdf2ee53ef46eb3663ff4b0d9bf4562-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 9fdf2ee53ef46eb3663ff4b0d9bf4562 2023-02-10 08:00:19,142 DEBUG [StoreOpener-9fdf2ee53ef46eb3663ff4b0d9bf4562-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/namespace/9fdf2ee53ef46eb3663ff4b0d9bf4562/info 2023-02-10 08:00:19,142 DEBUG [StoreOpener-9fdf2ee53ef46eb3663ff4b0d9bf4562-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/namespace/9fdf2ee53ef46eb3663ff4b0d9bf4562/info 2023-02-10 08:00:19,143 INFO [StoreOpener-9fdf2ee53ef46eb3663ff4b0d9bf4562-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9fdf2ee53ef46eb3663ff4b0d9bf4562 columnFamilyName info 2023-02-10 08:00:19,144 INFO [StoreOpener-9fdf2ee53ef46eb3663ff4b0d9bf4562-1] regionserver.HStore(310): Store=9fdf2ee53ef46eb3663ff4b0d9bf4562/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 08:00:19,145 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/namespace/9fdf2ee53ef46eb3663ff4b0d9bf4562 2023-02-10 08:00:19,146 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/namespace/9fdf2ee53ef46eb3663ff4b0d9bf4562 2023-02-10 08:00:19,149 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9fdf2ee53ef46eb3663ff4b0d9bf4562 2023-02-10 08:00:19,151 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/namespace/9fdf2ee53ef46eb3663ff4b0d9bf4562/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 08:00:19,152 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9fdf2ee53ef46eb3663ff4b0d9bf4562; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63098132, jitterRate=-0.059764564037323}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-10 08:00:19,152 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9fdf2ee53ef46eb3663ff4b0d9bf4562: Running coprocessor pre-open hook at 1676016019139Writing region info on filesystem at 1676016019139Initializing all the Stores at 1676016019140 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676016019140Cleaning up temporary data from old regions at 1676016019147 (+7 ms)Cleaning up detritus from prior splits at 1676016019147Running coprocessor post-open hooks at 1676016019152 (+5 ms)Region opened successfully at 1676016019152 2023-02-10 08:00:19,153 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562., pid=6, masterSystemTime=1676016019131 2023-02-10 08:00:19,157 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. 2023-02-10 08:00:19,158 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. 2023-02-10 08:00:19,158 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=9fdf2ee53ef46eb3663ff4b0d9bf4562, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:19,159 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676016019158"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676016019158"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676016019158"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676016019158"}]},"ts":"1676016019158"} 2023-02-10 08:00:19,164 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-10 08:00:19,165 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 9fdf2ee53ef46eb3663ff4b0d9bf4562, server=jenkins-hbase4.apache.org,46575,1676016018000 in 185 msec 2023-02-10 08:00:19,168 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-10 08:00:19,169 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=9fdf2ee53ef46eb3663ff4b0d9bf4562, ASSIGN in 346 msec 2023-02-10 08:00:19,170 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-10 08:00:19,170 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676016019170"}]},"ts":"1676016019170"} 2023-02-10 08:00:19,172 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-10 08:00:19,175 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-10 08:00:19,178 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 416 msec 2023-02-10 08:00:19,262 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-10 08:00:19,263 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-10 08:00:19,263 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:19,267 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-10 08:00:19,269 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44170, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-10 08:00:19,272 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-10 08:00:19,283 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-10 08:00:19,287 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 15 msec 2023-02-10 08:00:19,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-10 08:00:19,304 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-10 08:00:19,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 14 msec 2023-02-10 08:00:19,319 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-10 08:00:19,322 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-10 08:00:19,322 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 1.233sec 2023-02-10 08:00:19,322 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-10 08:00:19,322 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-10 08:00:19,322 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-10 08:00:19,322 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35963,1676016017945-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-10 08:00:19,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35963,1676016017945-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-10 08:00:19,324 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-10 08:00:19,380 DEBUG [Listener at localhost/34025] zookeeper.ReadOnlyZKClient(139): Connect 0x13815be5 to 127.0.0.1:57601 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 08:00:19,385 DEBUG [Listener at localhost/34025] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7ac40a55, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 08:00:19,387 DEBUG [hconnection-0x2ffcad7d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-10 08:00:19,389 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51372, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-10 08:00:19,391 INFO [Listener at localhost/34025] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,35963,1676016017945 2023-02-10 08:00:19,391 DEBUG [Listener at localhost/34025] zookeeper.ReadOnlyZKClient(139): Connect 0x70230111 to 127.0.0.1:57601 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 08:00:19,397 DEBUG [ReadOnlyZKClient-127.0.0.1:57601@0x70230111] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@751ee937, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 08:00:19,398 DEBUG [Listener at localhost/34025] client.ConnectionUtils(586): Start fetching master stub from registry 2023-02-10 08:00:19,399 DEBUG [ReadOnlyZKClient-127.0.0.1:57601@0x70230111] client.AsyncConnectionImpl(289): The fetched master address is jenkins-hbase4.apache.org,35963,1676016017945 2023-02-10 08:00:19,399 DEBUG [ReadOnlyZKClient-127.0.0.1:57601@0x70230111] client.ConnectionUtils(594): The fetched master stub is org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$Stub@7121734c 2023-02-10 08:00:19,402 DEBUG [ReadOnlyZKClient-127.0.0.1:57601@0x70230111] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-10 08:00:19,404 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57164, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-10 08:00:19,404 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35963] master.MasterRpcServices(1560): Client=jenkins//172.31.14.131 shutdown 2023-02-10 08:00:19,404 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35963] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,35963,1676016017945 2023-02-10 08:00:19,407 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-10 08:00:19,407 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35963] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 08:00:19,407 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-10 08:00:19,407 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-10 08:00:19,407 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-10 08:00:19,408 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35963] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x50a4843f to 127.0.0.1:57601 2023-02-10 08:00:19,408 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:19,408 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35963] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:19,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 08:00:19,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 08:00:19,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 08:00:19,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 08:00:19,469 INFO [RS:1;jenkins-hbase4:43847] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43847,1676016018030' ***** 2023-02-10 08:00:19,469 INFO [RS:1;jenkins-hbase4:43847] regionserver.HRegionServer(2310): STOPPED: Exiting; cluster shutdown set and not carrying any regions 2023-02-10 08:00:19,470 INFO [RS:1;jenkins-hbase4:43847] regionserver.HeapMemoryManager(220): Stopping 2023-02-10 08:00:19,470 INFO [RS:1;jenkins-hbase4:43847] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-10 08:00:19,470 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-10 08:00:19,470 INFO [RS:1;jenkins-hbase4:43847] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-10 08:00:19,471 INFO [RS:1;jenkins-hbase4:43847] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43847,1676016018030 2023-02-10 08:00:19,471 DEBUG [RS:1;jenkins-hbase4:43847] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4adee753 to 127.0.0.1:57601 2023-02-10 08:00:19,471 DEBUG [RS:1;jenkins-hbase4:43847] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:19,471 INFO [RS:1;jenkins-hbase4:43847] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43847,1676016018030; all regions closed. 2023-02-10 08:00:19,474 INFO [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(1065): Closing user regions 2023-02-10 08:00:19,474 INFO [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(1065): Closing user regions 2023-02-10 08:00:19,474 INFO [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(3304): Received CLOSE for 9fdf2ee53ef46eb3663ff4b0d9bf4562 2023-02-10 08:00:19,477 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9fdf2ee53ef46eb3663ff4b0d9bf4562, disabling compactions & flushes 2023-02-10 08:00:19,477 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. 2023-02-10 08:00:19,477 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. 2023-02-10 08:00:19,477 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. after waiting 0 ms 2023-02-10 08:00:19,477 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. 2023-02-10 08:00:19,477 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 9fdf2ee53ef46eb3663ff4b0d9bf4562 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-10 08:00:19,480 DEBUG [RS:1;jenkins-hbase4:43847] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/oldWALs 2023-02-10 08:00:19,480 INFO [RS:1;jenkins-hbase4:43847] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43847%2C1676016018030:(num 1676016018504) 2023-02-10 08:00:19,480 DEBUG [RS:1;jenkins-hbase4:43847] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:19,480 INFO [RS:1;jenkins-hbase4:43847] regionserver.LeaseManager(133): Closed leases 2023-02-10 08:00:19,481 INFO [RS:1;jenkins-hbase4:43847] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-10 08:00:19,481 INFO [RS:1;jenkins-hbase4:43847] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-10 08:00:19,481 INFO [RS:1;jenkins-hbase4:43847] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-10 08:00:19,481 INFO [RS:1;jenkins-hbase4:43847] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-10 08:00:19,481 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-10 08:00:19,482 INFO [RS:1;jenkins-hbase4:43847] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43847 2023-02-10 08:00:19,486 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43847,1676016018030 2023-02-10 08:00:19,486 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43847,1676016018030 2023-02-10 08:00:19,486 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43847,1676016018030 2023-02-10 08:00:19,486 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:19,486 ERROR [Listener at localhost/34025-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7d1a2b80 rejected from java.util.concurrent.ThreadPoolExecutor@7925a0fb[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:19,486 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:19,486 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:19,487 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:19,487 ERROR [Listener at localhost/34025-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@21bb3427 rejected from java.util.concurrent.ThreadPoolExecutor@7925a0fb[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:19,489 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43847,1676016018030] 2023-02-10 08:00:19,490 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43847,1676016018030; numProcessing=1 2023-02-10 08:00:19,490 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:19,490 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:19,492 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,43847,1676016018030 already deleted, retry=false 2023-02-10 08:00:19,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:19,492 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43847,1676016018030 expired; onlineServers=2 2023-02-10 08:00:19,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:19,493 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,43847,1676016018030 znode expired, triggering replicatorRemoved event 2023-02-10 08:00:19,493 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,43847,1676016018030 znode expired, triggering replicatorRemoved event 2023-02-10 08:00:19,494 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:19,494 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:19,494 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:19,495 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:19,500 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/namespace/9fdf2ee53ef46eb3663ff4b0d9bf4562/.tmp/info/ec70c502341e432ba80ea58ae73eba20 2023-02-10 08:00:19,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/namespace/9fdf2ee53ef46eb3663ff4b0d9bf4562/.tmp/info/ec70c502341e432ba80ea58ae73eba20 as hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/namespace/9fdf2ee53ef46eb3663ff4b0d9bf4562/info/ec70c502341e432ba80ea58ae73eba20 2023-02-10 08:00:19,515 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/namespace/9fdf2ee53ef46eb3663ff4b0d9bf4562/info/ec70c502341e432ba80ea58ae73eba20, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 08:00:19,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 9fdf2ee53ef46eb3663ff4b0d9bf4562 in 39ms, sequenceid=6, compaction requested=false 2023-02-10 08:00:19,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/namespace/9fdf2ee53ef46eb3663ff4b0d9bf4562/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-10 08:00:19,533 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. 2023-02-10 08:00:19,533 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9fdf2ee53ef46eb3663ff4b0d9bf4562: Waiting for close lock at 1676016019475Running coprocessor pre-close hooks at 1676016019475Disabling compacts and flushes for region at 1676016019475Disabling writes for close at 1676016019477 (+2 ms)Obtaining lock to block concurrent updates at 1676016019477Preparing flush snapshotting stores in 9fdf2ee53ef46eb3663ff4b0d9bf4562 at 1676016019477Finished memstore snapshotting hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676016019477Flushing stores of hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. at 1676016019478 (+1 ms)Flushing 9fdf2ee53ef46eb3663ff4b0d9bf4562/info: creating writer at 1676016019478Flushing 9fdf2ee53ef46eb3663ff4b0d9bf4562/info: appending metadata at 1676016019485 (+7 ms)Flushing 9fdf2ee53ef46eb3663ff4b0d9bf4562/info: closing flushed file at 1676016019485Flushing 9fdf2ee53ef46eb3663ff4b0d9bf4562/info: reopening flushed file at 1676016019509 (+24 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 9fdf2ee53ef46eb3663ff4b0d9bf4562 in 39ms, sequenceid=6, compaction requested=false at 1676016019516 (+7 ms)Writing region close event to WAL at 1676016019519 (+3 ms)Running coprocessor post-close hooks at 1676016019532 (+13 ms)Closed at 1676016019533 (+1 ms) 2023-02-10 08:00:19,534 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=35963] assignment.AssignmentManager(1094): RegionServer CLOSED 9fdf2ee53ef46eb3663ff4b0d9bf4562 2023-02-10 08:00:19,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676016018759.9fdf2ee53ef46eb3663ff4b0d9bf4562. 2023-02-10 08:00:19,569 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-10 08:00:19,578 DEBUG [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(1084): Waiting on 1588230740 2023-02-10 08:00:19,578 INFO [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,46575,1676016018000' ***** 2023-02-10 08:00:19,578 INFO [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(2310): STOPPED: Exiting; cluster shutdown set and not carrying any regions 2023-02-10 08:00:19,580 INFO [RS:0;jenkins-hbase4:46575] regionserver.HeapMemoryManager(220): Stopping 2023-02-10 08:00:19,580 INFO [RS:0;jenkins-hbase4:46575] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-10 08:00:19,580 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-10 08:00:19,580 INFO [RS:0;jenkins-hbase4:46575] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-10 08:00:19,580 INFO [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:19,580 DEBUG [RS:0;jenkins-hbase4:46575] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x425b4345 to 127.0.0.1:57601 2023-02-10 08:00:19,581 DEBUG [RS:0;jenkins-hbase4:46575] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:19,581 INFO [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,46575,1676016018000; all regions closed. 2023-02-10 08:00:19,588 DEBUG [RS:0;jenkins-hbase4:46575] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/oldWALs 2023-02-10 08:00:19,588 INFO [RS:0;jenkins-hbase4:46575] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46575%2C1676016018000:(num 1676016018513) 2023-02-10 08:00:19,588 DEBUG [RS:0;jenkins-hbase4:46575] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:19,588 INFO [RS:0;jenkins-hbase4:46575] regionserver.LeaseManager(133): Closed leases 2023-02-10 08:00:19,588 INFO [RS:0;jenkins-hbase4:46575] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-10 08:00:19,588 INFO [RS:0;jenkins-hbase4:46575] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-10 08:00:19,588 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-10 08:00:19,588 INFO [RS:0;jenkins-hbase4:46575] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-10 08:00:19,588 INFO [RS:0;jenkins-hbase4:46575] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-10 08:00:19,589 INFO [RS:0;jenkins-hbase4:46575] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46575 2023-02-10 08:00:19,593 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:19,593 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:19,593 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:19,593 ERROR [Listener at localhost/34025-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@122680b rejected from java.util.concurrent.ThreadPoolExecutor@51decf7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:19,593 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46575,1676016018000 2023-02-10 08:00:19,594 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:19,596 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46575,1676016018000] 2023-02-10 08:00:19,596 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46575,1676016018000; numProcessing=2 2023-02-10 08:00:19,597 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,46575,1676016018000 already deleted, retry=false 2023-02-10 08:00:19,597 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46575,1676016018000 expired; onlineServers=1 2023-02-10 08:00:19,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:19,598 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,46575,1676016018000 znode expired, triggering replicatorRemoved event 2023-02-10 08:00:19,598 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:19,610 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:19,610 INFO [RS:1;jenkins-hbase4:43847] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43847,1676016018030; zookeeper connection closed. 2023-02-10 08:00:19,610 ERROR [Listener at localhost/34025-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@59271692 rejected from java.util.concurrent.ThreadPoolExecutor@7925a0fb[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:19,610 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@a3e8d9a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@a3e8d9a 2023-02-10 08:00:19,610 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:43847-0x1003d3ecbc50002, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:19,611 ERROR [Listener at localhost/34025-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@24ead35e rejected from java.util.concurrent.ThreadPoolExecutor@7925a0fb[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:19,674 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-10 08:00:19,680 INFO [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,34017,1676016018064' ***** 2023-02-10 08:00:19,680 INFO [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(2310): STOPPED: Stopped; only catalog regions remaining online 2023-02-10 08:00:19,680 INFO [RS:2;jenkins-hbase4:34017] regionserver.HeapMemoryManager(220): Stopping 2023-02-10 08:00:19,680 INFO [RS:2;jenkins-hbase4:34017] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-10 08:00:19,680 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-10 08:00:19,680 INFO [RS:2;jenkins-hbase4:34017] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-10 08:00:19,681 INFO [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:19,681 DEBUG [RS:2;jenkins-hbase4:34017] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x74756ad6 to 127.0.0.1:57601 2023-02-10 08:00:19,681 DEBUG [RS:2;jenkins-hbase4:34017] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:19,681 INFO [RS:2;jenkins-hbase4:34017] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-10 08:00:19,681 INFO [RS:2;jenkins-hbase4:34017] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-10 08:00:19,681 INFO [RS:2;jenkins-hbase4:34017] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-10 08:00:19,681 INFO [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-10 08:00:19,681 INFO [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-10 08:00:19,681 DEBUG [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-10 08:00:19,682 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-10 08:00:19,682 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-10 08:00:19,682 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-10 08:00:19,682 DEBUG [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-10 08:00:19,682 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-10 08:00:19,683 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-10 08:00:19,683 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-10 08:00:19,683 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-02-10 08:00:19,700 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/.tmp/info/a22bfc2b415f4d52adf476e1c2d7f29d 2023-02-10 08:00:19,722 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/.tmp/table/bfc8e1371763493aa9121159369ce102 2023-02-10 08:00:19,729 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/.tmp/info/a22bfc2b415f4d52adf476e1c2d7f29d as hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/info/a22bfc2b415f4d52adf476e1c2d7f29d 2023-02-10 08:00:19,736 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/info/a22bfc2b415f4d52adf476e1c2d7f29d, entries=10, sequenceid=9, filesize=5.9 K 2023-02-10 08:00:19,737 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/.tmp/table/bfc8e1371763493aa9121159369ce102 as hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/table/bfc8e1371763493aa9121159369ce102 2023-02-10 08:00:19,746 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/table/bfc8e1371763493aa9121159369ce102, entries=2, sequenceid=9, filesize=4.7 K 2023-02-10 08:00:19,747 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 64ms, sequenceid=9, compaction requested=false 2023-02-10 08:00:19,755 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-02-10 08:00:19,755 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-10 08:00:19,755 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-10 08:00:19,755 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676016019681Running coprocessor pre-close hooks at 1676016019681Disabling compacts and flushes for region at 1676016019681Disabling writes for close at 1676016019683 (+2 ms)Obtaining lock to block concurrent updates at 1676016019683Preparing flush snapshotting stores in 1588230740 at 1676016019683Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=1290, getHeapSize=2912, getOffHeapSize=0, getCellsCount=12 at 1676016019683Flushing stores of hbase:meta,,1.1588230740 at 1676016019684 (+1 ms)Flushing 1588230740/info: creating writer at 1676016019684Flushing 1588230740/info: appending metadata at 1676016019688 (+4 ms)Flushing 1588230740/info: closing flushed file at 1676016019688Flushing 1588230740/table: creating writer at 1676016019709 (+21 ms)Flushing 1588230740/table: appending metadata at 1676016019712 (+3 ms)Flushing 1588230740/table: closing flushed file at 1676016019712Flushing 1588230740/info: reopening flushed file at 1676016019730 (+18 ms)Flushing 1588230740/table: reopening flushed file at 1676016019738 (+8 ms)Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 64ms, sequenceid=9, compaction requested=false at 1676016019747 (+9 ms)Writing region close event to WAL at 1676016019751 (+4 ms)Running coprocessor post-close hooks at 1676016019755 (+4 ms)Closed at 1676016019755 2023-02-10 08:00:19,756 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-10 08:00:19,883 INFO [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,34017,1676016018064; all regions closed. 2023-02-10 08:00:19,889 DEBUG [RS:2;jenkins-hbase4:34017] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/oldWALs 2023-02-10 08:00:19,889 INFO [RS:2;jenkins-hbase4:34017] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34017%2C1676016018064.meta:.meta(num 1676016018680) 2023-02-10 08:00:19,895 DEBUG [RS:2;jenkins-hbase4:34017] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/oldWALs 2023-02-10 08:00:19,895 INFO [RS:2;jenkins-hbase4:34017] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34017%2C1676016018064:(num 1676016018513) 2023-02-10 08:00:19,895 DEBUG [RS:2;jenkins-hbase4:34017] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:19,895 INFO [RS:2;jenkins-hbase4:34017] regionserver.LeaseManager(133): Closed leases 2023-02-10 08:00:19,895 INFO [RS:2;jenkins-hbase4:34017] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-10 08:00:19,895 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-10 08:00:19,896 INFO [RS:2;jenkins-hbase4:34017] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34017 2023-02-10 08:00:19,899 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34017,1676016018064 2023-02-10 08:00:19,899 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:19,899 ERROR [Listener at localhost/34025-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7d7b2da2 rejected from java.util.concurrent.ThreadPoolExecutor@7944b6d5[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:19,899 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 08:00:19,899 ERROR [Listener at localhost/34025-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@29829441 rejected from java.util.concurrent.ThreadPoolExecutor@7944b6d5[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:19,901 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34017,1676016018064] 2023-02-10 08:00:19,901 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34017,1676016018064; numProcessing=3 2023-02-10 08:00:19,902 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34017,1676016018064 already deleted, retry=false 2023-02-10 08:00:19,902 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34017,1676016018064 expired; onlineServers=0 2023-02-10 08:00:19,902 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,35963,1676016017945' ***** 2023-02-10 08:00:19,902 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-10 08:00:19,903 DEBUG [M:0;jenkins-hbase4:35963] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5c6e0aa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-10 08:00:19,903 INFO [M:0;jenkins-hbase4:35963] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,35963,1676016017945 2023-02-10 08:00:19,903 INFO [M:0;jenkins-hbase4:35963] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,35963,1676016017945; all regions closed. 2023-02-10 08:00:19,903 DEBUG [M:0;jenkins-hbase4:35963] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:19,903 DEBUG [M:0;jenkins-hbase4:35963] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-10 08:00:19,903 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-10 08:00:19,903 DEBUG [M:0;jenkins-hbase4:35963] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-10 08:00:19,903 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676016018257] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676016018257,5,FailOnTimeoutGroup] 2023-02-10 08:00:19,903 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676016018257] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676016018257,5,FailOnTimeoutGroup] 2023-02-10 08:00:19,904 INFO [M:0;jenkins-hbase4:35963] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-10 08:00:19,904 INFO [M:0;jenkins-hbase4:35963] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-10 08:00:19,905 INFO [M:0;jenkins-hbase4:35963] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-10 08:00:19,905 DEBUG [M:0;jenkins-hbase4:35963] master.HMaster(1502): Stopping service threads 2023-02-10 08:00:19,905 INFO [M:0;jenkins-hbase4:35963] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-10 08:00:19,905 ERROR [M:0;jenkins-hbase4:35963] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] 2023-02-10 08:00:19,905 INFO [M:0;jenkins-hbase4:35963] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 08:00:19,905 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-10 08:00:19,906 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-10 08:00:19,906 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 08:00:19,906 DEBUG [M:0;jenkins-hbase4:35963] zookeeper.ZKUtil(398): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-10 08:00:19,906 WARN [M:0;jenkins-hbase4:35963] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-10 08:00:19,906 INFO [M:0;jenkins-hbase4:35963] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 08:00:19,906 INFO [M:0;jenkins-hbase4:35963] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-10 08:00:19,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 08:00:19,907 DEBUG [M:0;jenkins-hbase4:35963] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-10 08:00:19,907 INFO [M:0;jenkins-hbase4:35963] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:19,907 DEBUG [M:0;jenkins-hbase4:35963] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:19,907 DEBUG [M:0;jenkins-hbase4:35963] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-10 08:00:19,907 DEBUG [M:0;jenkins-hbase4:35963] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:19,907 INFO [M:0;jenkins-hbase4:35963] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-02-10 08:00:19,911 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:19,911 INFO [RS:0;jenkins-hbase4:46575] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,46575,1676016018000; zookeeper connection closed. 2023-02-10 08:00:19,911 ERROR [Listener at localhost/34025-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@63650c0b rejected from java.util.concurrent.ThreadPoolExecutor@51decf7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:19,911 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3c228c1f] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3c228c1f 2023-02-10 08:00:19,911 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:46575-0x1003d3ecbc50001, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:19,911 ERROR [Listener at localhost/34025-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1b8572dd rejected from java.util.concurrent.ThreadPoolExecutor@51decf7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:19,912 INFO [Listener at localhost/34025] client.AsyncConnectionImpl(207): Connection has been closed by Listener at localhost/34025. 2023-02-10 08:00:19,912 DEBUG [Listener at localhost/34025] client.AsyncConnectionImpl(232): Call stack: at java.lang.Thread.getStackTrace(Thread.java:1564) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:209) at org.apache.hbase.thirdparty.com.google.common.io.Closeables.close(Closeables.java:79) at org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2.tearDown(TestAsyncClusterAdminApi2.java:75) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2023-02-10 08:00:19,913 DEBUG [Listener at localhost/34025] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:19,913 DEBUG [Listener at localhost/34025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x70230111 to 127.0.0.1:57601 2023-02-10 08:00:19,914 INFO [Listener at localhost/34025] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-10 08:00:19,914 DEBUG [Listener at localhost/34025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x13815be5 to 127.0.0.1:57601 2023-02-10 08:00:19,914 DEBUG [Listener at localhost/34025] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 08:00:19,915 DEBUG [Listener at localhost/34025] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-10 08:00:19,928 INFO [M:0;jenkins-hbase4:35963] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/953f5adf9e4b42e29316942a39565cef 2023-02-10 08:00:19,936 DEBUG [M:0;jenkins-hbase4:35963] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/953f5adf9e4b42e29316942a39565cef as hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/953f5adf9e4b42e29316942a39565cef 2023-02-10 08:00:19,942 INFO [M:0;jenkins-hbase4:35963] regionserver.HStore(1080): Added hdfs://localhost:40139/user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/953f5adf9e4b42e29316942a39565cef, entries=8, sequenceid=66, filesize=6.3 K 2023-02-10 08:00:19,943 INFO [M:0;jenkins-hbase4:35963] regionserver.HRegion(2947): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 36ms, sequenceid=66, compaction requested=false 2023-02-10 08:00:19,945 INFO [M:0;jenkins-hbase4:35963] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 08:00:19,945 DEBUG [M:0;jenkins-hbase4:35963] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676016019907Disabling compacts and flushes for region at 1676016019907Disabling writes for close at 1676016019907Obtaining lock to block concurrent updates at 1676016019907Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676016019907Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=24646, getHeapSize=30248, getOffHeapSize=0, getCellsCount=71 at 1676016019907Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676016019908 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676016019909 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676016019915 (+6 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676016019915Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676016019937 (+22 ms)Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 36ms, sequenceid=66, compaction requested=false at 1676016019943 (+6 ms)Writing region close event to WAL at 1676016019944 (+1 ms)Closed at 1676016019944 2023-02-10 08:00:19,948 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/6d120971-1e2d-ff3c-8b19-fa97df8c9af7/MasterData/WALs/jenkins-hbase4.apache.org,35963,1676016017945/jenkins-hbase4.apache.org%2C35963%2C1676016017945.1676016018170 not finished, retry = 0 2023-02-10 08:00:20,034 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-02-10 08:00:20,049 INFO [M:0;jenkins-hbase4:35963] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-10 08:00:20,049 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-10 08:00:20,050 INFO [M:0;jenkins-hbase4:35963] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35963 2023-02-10 08:00:20,052 DEBUG [M:0;jenkins-hbase4:35963] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,35963,1676016017945 already deleted, retry=false 2023-02-10 08:00:20,312 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:20,312 INFO [RS:2;jenkins-hbase4:34017] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,34017,1676016018064; zookeeper connection closed. 2023-02-10 08:00:20,312 ERROR [Listener at localhost/34025-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@519e5793 rejected from java.util.concurrent.ThreadPoolExecutor@7944b6d5[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:20,313 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6a451f3e] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6a451f3e 2023-02-10 08:00:20,313 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): regionserver:34017-0x1003d3ecbc50003, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:20,313 ERROR [Listener at localhost/34025-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@61746f86 rejected from java.util.concurrent.ThreadPoolExecutor@7944b6d5[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:20,313 INFO [Listener at localhost/34025] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-10 08:00:20,412 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:20,412 INFO [M:0;jenkins-hbase4:35963] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,35963,1676016017945; zookeeper connection closed. 2023-02-10 08:00:20,413 ERROR [Listener at localhost/34025-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5a604f27 rejected from java.util.concurrent.ThreadPoolExecutor@5e5fd0b3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 28] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:20,413 DEBUG [Listener at localhost/34025-EventThread] zookeeper.ZKWatcher(600): master:35963-0x1003d3ecbc50000, quorum=127.0.0.1:57601, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 08:00:20,413 ERROR [Listener at localhost/34025-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@724d8c17 rejected from java.util.concurrent.ThreadPoolExecutor@5e5fd0b3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 28] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 08:00:20,413 WARN [Listener at localhost/34025] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-10 08:00:20,418 INFO [Listener at localhost/34025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-10 08:00:20,522 WARN [BP-1222756245-172.31.14.131-1676016017189 heartbeating to localhost/127.0.0.1:40139] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-10 08:00:20,522 WARN [BP-1222756245-172.31.14.131-1676016017189 heartbeating to localhost/127.0.0.1:40139] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1222756245-172.31.14.131-1676016017189 (Datanode Uuid 980b38c3-e4f8-44c7-af10-c0b1aa20744e) service to localhost/127.0.0.1:40139 2023-02-10 08:00:20,523 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/cluster_5d1ed0ef-257f-d997-688c-9c855b3f8051/dfs/data/data5/current/BP-1222756245-172.31.14.131-1676016017189] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 08:00:20,524 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/cluster_5d1ed0ef-257f-d997-688c-9c855b3f8051/dfs/data/data6/current/BP-1222756245-172.31.14.131-1676016017189] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 08:00:20,525 WARN [Listener at localhost/34025] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-10 08:00:20,528 INFO [Listener at localhost/34025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-10 08:00:20,632 WARN [BP-1222756245-172.31.14.131-1676016017189 heartbeating to localhost/127.0.0.1:40139] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-10 08:00:20,632 WARN [BP-1222756245-172.31.14.131-1676016017189 heartbeating to localhost/127.0.0.1:40139] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1222756245-172.31.14.131-1676016017189 (Datanode Uuid 097a8492-c84c-44fa-91d0-5720bb534803) service to localhost/127.0.0.1:40139 2023-02-10 08:00:20,632 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/cluster_5d1ed0ef-257f-d997-688c-9c855b3f8051/dfs/data/data3/current/BP-1222756245-172.31.14.131-1676016017189] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 08:00:20,633 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/cluster_5d1ed0ef-257f-d997-688c-9c855b3f8051/dfs/data/data4/current/BP-1222756245-172.31.14.131-1676016017189] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 08:00:20,634 WARN [Listener at localhost/34025] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-10 08:00:20,637 INFO [Listener at localhost/34025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-10 08:00:20,741 WARN [BP-1222756245-172.31.14.131-1676016017189 heartbeating to localhost/127.0.0.1:40139] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-10 08:00:20,741 WARN [BP-1222756245-172.31.14.131-1676016017189 heartbeating to localhost/127.0.0.1:40139] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1222756245-172.31.14.131-1676016017189 (Datanode Uuid 54fe9603-238a-48ff-b4bf-c3f0a384c122) service to localhost/127.0.0.1:40139 2023-02-10 08:00:20,742 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/cluster_5d1ed0ef-257f-d997-688c-9c855b3f8051/dfs/data/data1/current/BP-1222756245-172.31.14.131-1676016017189] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 08:00:20,742 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/6df8b2d9-c594-ea69-6193-9e53e6f9d7b9/cluster_5d1ed0ef-257f-d997-688c-9c855b3f8051/dfs/data/data2/current/BP-1222756245-172.31.14.131-1676016017189] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 08:00:20,754 INFO [Listener at localhost/34025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-10 08:00:20,868 INFO [Listener at localhost/34025] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-10 08:00:20,893 INFO [Listener at localhost/34025] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-02-10 08:00:20,905 INFO [Listener at localhost/34025] hbase.ResourceChecker(175): after: client.TestAsyncClusterAdminApi2#testShutdown Thread=107 (was 77) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (267790143) connection to localhost/127.0.0.1:40139 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:40139 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:40139 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/34025 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (267790143) connection to localhost/127.0.0.1:40139 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (267790143) connection to localhost/127.0.0.1:40139 from jenkins.hfs.4 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (267790143) connection to localhost/127.0.0.1:40139 from jenkins.hfs.3 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (267790143) connection to localhost/127.0.0.1:40139 from jenkins.hfs.5 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:40139 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:40139 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=541 (was 500) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=264 (was 288), ProcessCount=168 (was 168), AvailableMemoryMB=3399 (was 3536)