2023-02-11 15:00:39,986 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459 2023-02-11 15:00:40,001 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2 timeout: 13 mins 2023-02-11 15:00:40,039 INFO [Time-limited test] hbase.ResourceChecker(147): before: client.TestAsyncClusterAdminApi2#testStop Thread=8, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=239, ProcessCount=168, AvailableMemoryMB=4246 2023-02-11 15:00:40,046 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-11 15:00:40,046 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/cluster_481e5d89-aaa2-47ab-8331-fe2aab94c5ba, deleteOnExit=true 2023-02-11 15:00:40,047 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-11 15:00:40,047 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/test.cache.data in system properties and HBase conf 2023-02-11 15:00:40,048 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/hadoop.tmp.dir in system properties and HBase conf 2023-02-11 15:00:40,049 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/hadoop.log.dir in system properties and HBase conf 2023-02-11 15:00:40,049 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-11 15:00:40,050 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-11 15:00:40,050 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-11 15:00:40,163 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-11 15:00:40,533 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-11 15:00:40,537 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-11 15:00:40,537 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-11 15:00:40,538 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-11 15:00:40,538 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-11 15:00:40,538 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-11 15:00:40,538 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-11 15:00:40,539 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-11 15:00:40,539 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-11 15:00:40,539 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-11 15:00:40,540 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/nfs.dump.dir in system properties and HBase conf 2023-02-11 15:00:40,540 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/java.io.tmpdir in system properties and HBase conf 2023-02-11 15:00:40,540 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-11 15:00:40,540 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-11 15:00:40,541 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-11 15:00:40,996 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-11 15:00:41,000 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-11 15:00:41,269 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-11 15:00:41,426 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-11 15:00:41,440 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 15:00:41,476 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 15:00:41,507 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/java.io.tmpdir/Jetty_localhost_42159_hdfs____hy93wh/webapp 2023-02-11 15:00:41,658 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42159 2023-02-11 15:00:41,668 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-11 15:00:41,668 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-11 15:00:42,043 WARN [Listener at localhost/38847] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 15:00:42,143 WARN [Listener at localhost/38847] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 15:00:42,165 WARN [Listener at localhost/38847] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 15:00:42,171 INFO [Listener at localhost/38847] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 15:00:42,175 INFO [Listener at localhost/38847] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/java.io.tmpdir/Jetty_localhost_40873_datanode____.2jrojl/webapp 2023-02-11 15:00:42,269 INFO [Listener at localhost/38847] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40873 2023-02-11 15:00:42,577 WARN [Listener at localhost/36793] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 15:00:42,589 WARN [Listener at localhost/36793] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 15:00:42,592 WARN [Listener at localhost/36793] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 15:00:42,595 INFO [Listener at localhost/36793] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 15:00:42,604 INFO [Listener at localhost/36793] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/java.io.tmpdir/Jetty_localhost_46793_datanode____.ntyk7e/webapp 2023-02-11 15:00:42,710 INFO [Listener at localhost/36793] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46793 2023-02-11 15:00:42,719 WARN [Listener at localhost/34345] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 15:00:42,733 WARN [Listener at localhost/34345] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 15:00:42,737 WARN [Listener at localhost/34345] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 15:00:42,739 INFO [Listener at localhost/34345] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 15:00:42,743 INFO [Listener at localhost/34345] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/java.io.tmpdir/Jetty_localhost_35085_datanode____.jnvv2/webapp 2023-02-11 15:00:42,850 INFO [Listener at localhost/34345] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35085 2023-02-11 15:00:42,861 WARN [Listener at localhost/36191] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 15:00:43,060 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe8799447ce4de644: Processing first storage report for DS-c12805c7-d335-4d6f-b427-bcd385ac8afa from datanode eeb91041-2f31-4b32-9e1c-18e651680daa 2023-02-11 15:00:43,062 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe8799447ce4de644: from storage DS-c12805c7-d335-4d6f-b427-bcd385ac8afa node DatanodeRegistration(127.0.0.1:42823, datanodeUuid=eeb91041-2f31-4b32-9e1c-18e651680daa, infoPort=34913, infoSecurePort=0, ipcPort=34345, storageInfo=lv=-57;cid=testClusterID;nsid=112107650;c=1676127641071), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-11 15:00:43,062 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x55bd4ab1ba30f6a8: Processing first storage report for DS-ad5ccbfc-2075-4a9a-acab-951204632a80 from datanode cfa573b9-160e-4f69-9a79-1d6bc607406f 2023-02-11 15:00:43,063 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x55bd4ab1ba30f6a8: from storage DS-ad5ccbfc-2075-4a9a-acab-951204632a80 node DatanodeRegistration(127.0.0.1:40405, datanodeUuid=cfa573b9-160e-4f69-9a79-1d6bc607406f, infoPort=46827, infoSecurePort=0, ipcPort=36793, storageInfo=lv=-57;cid=testClusterID;nsid=112107650;c=1676127641071), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 15:00:43,063 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6259bed38668a652: Processing first storage report for DS-f39f2519-553b-44cd-943f-211e5b0b456a from datanode f1797d6a-658b-43e2-aba7-ea82a7c47cab 2023-02-11 15:00:43,063 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6259bed38668a652: from storage DS-f39f2519-553b-44cd-943f-211e5b0b456a node DatanodeRegistration(127.0.0.1:44679, datanodeUuid=f1797d6a-658b-43e2-aba7-ea82a7c47cab, infoPort=33367, infoSecurePort=0, ipcPort=36191, storageInfo=lv=-57;cid=testClusterID;nsid=112107650;c=1676127641071), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 15:00:43,063 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe8799447ce4de644: Processing first storage report for DS-4deaaa88-5698-471c-9109-28e4d9e04c12 from datanode eeb91041-2f31-4b32-9e1c-18e651680daa 2023-02-11 15:00:43,063 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe8799447ce4de644: from storage DS-4deaaa88-5698-471c-9109-28e4d9e04c12 node DatanodeRegistration(127.0.0.1:42823, datanodeUuid=eeb91041-2f31-4b32-9e1c-18e651680daa, infoPort=34913, infoSecurePort=0, ipcPort=34345, storageInfo=lv=-57;cid=testClusterID;nsid=112107650;c=1676127641071), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 15:00:43,064 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x55bd4ab1ba30f6a8: Processing first storage report for DS-d54b07e6-5884-48a4-aab6-35d04c639306 from datanode cfa573b9-160e-4f69-9a79-1d6bc607406f 2023-02-11 15:00:43,064 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x55bd4ab1ba30f6a8: from storage DS-d54b07e6-5884-48a4-aab6-35d04c639306 node DatanodeRegistration(127.0.0.1:40405, datanodeUuid=cfa573b9-160e-4f69-9a79-1d6bc607406f, infoPort=46827, infoSecurePort=0, ipcPort=36793, storageInfo=lv=-57;cid=testClusterID;nsid=112107650;c=1676127641071), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 15:00:43,064 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6259bed38668a652: Processing first storage report for DS-38b09769-f5ea-4031-a00e-8f09b80e3e16 from datanode f1797d6a-658b-43e2-aba7-ea82a7c47cab 2023-02-11 15:00:43,064 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6259bed38668a652: from storage DS-38b09769-f5ea-4031-a00e-8f09b80e3e16 node DatanodeRegistration(127.0.0.1:44679, datanodeUuid=f1797d6a-658b-43e2-aba7-ea82a7c47cab, infoPort=33367, infoSecurePort=0, ipcPort=36191, storageInfo=lv=-57;cid=testClusterID;nsid=112107650;c=1676127641071), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 15:00:43,258 DEBUG [Listener at localhost/36191] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459 2023-02-11 15:00:43,320 INFO [Listener at localhost/36191] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/cluster_481e5d89-aaa2-47ab-8331-fe2aab94c5ba/zookeeper_0, clientPort=63275, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/cluster_481e5d89-aaa2-47ab-8331-fe2aab94c5ba/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/cluster_481e5d89-aaa2-47ab-8331-fe2aab94c5ba/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-11 15:00:43,333 INFO [Listener at localhost/36191] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=63275 2023-02-11 15:00:43,345 INFO [Listener at localhost/36191] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:43,348 INFO [Listener at localhost/36191] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:44,014 INFO [Listener at localhost/36191] util.FSUtils(479): Created version file at hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46 with version=8 2023-02-11 15:00:44,014 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/hbase-staging 2023-02-11 15:00:44,316 INFO [Listener at localhost/36191] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-11 15:00:44,760 INFO [Listener at localhost/36191] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=6 2023-02-11 15:00:44,791 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:44,791 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:44,792 INFO [Listener at localhost/36191] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 15:00:44,792 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:44,792 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 15:00:44,929 INFO [Listener at localhost/36191] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 15:00:44,998 DEBUG [Listener at localhost/36191] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-11 15:00:45,095 INFO [Listener at localhost/36191] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37295 2023-02-11 15:00:45,105 INFO [Listener at localhost/36191] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:45,107 INFO [Listener at localhost/36191] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:45,128 INFO [Listener at localhost/36191] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37295 connecting to ZooKeeper ensemble=127.0.0.1:63275 2023-02-11 15:00:45,169 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:372950x0, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 15:00:45,171 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:37295-0x10043e6106d0000 connected 2023-02-11 15:00:45,194 DEBUG [Listener at localhost/36191] zookeeper.ZKUtil(164): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 15:00:45,195 DEBUG [Listener at localhost/36191] zookeeper.ZKUtil(164): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 15:00:45,198 DEBUG [Listener at localhost/36191] zookeeper.ZKUtil(164): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 15:00:45,206 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37295 2023-02-11 15:00:45,206 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37295 2023-02-11 15:00:45,207 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37295 2023-02-11 15:00:45,207 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37295 2023-02-11 15:00:45,207 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37295 2023-02-11 15:00:45,213 INFO [Listener at localhost/36191] master.HMaster(439): hbase.rootdir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46, hbase.cluster.distributed=false 2023-02-11 15:00:45,278 INFO [Listener at localhost/36191] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-11 15:00:45,278 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:45,279 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:45,279 INFO [Listener at localhost/36191] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 15:00:45,279 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:45,279 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 15:00:45,284 INFO [Listener at localhost/36191] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 15:00:45,286 INFO [Listener at localhost/36191] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33713 2023-02-11 15:00:45,289 INFO [Listener at localhost/36191] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 15:00:45,294 DEBUG [Listener at localhost/36191] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 15:00:45,296 INFO [Listener at localhost/36191] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:45,297 INFO [Listener at localhost/36191] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:45,299 INFO [Listener at localhost/36191] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33713 connecting to ZooKeeper ensemble=127.0.0.1:63275 2023-02-11 15:00:45,303 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:337130x0, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 15:00:45,304 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33713-0x10043e6106d0001 connected 2023-02-11 15:00:45,304 DEBUG [Listener at localhost/36191] zookeeper.ZKUtil(164): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 15:00:45,305 DEBUG [Listener at localhost/36191] zookeeper.ZKUtil(164): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 15:00:45,306 DEBUG [Listener at localhost/36191] zookeeper.ZKUtil(164): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 15:00:45,307 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33713 2023-02-11 15:00:45,307 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33713 2023-02-11 15:00:45,307 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33713 2023-02-11 15:00:45,308 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33713 2023-02-11 15:00:45,308 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33713 2023-02-11 15:00:45,321 INFO [Listener at localhost/36191] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-11 15:00:45,322 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:45,322 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:45,322 INFO [Listener at localhost/36191] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 15:00:45,322 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:45,323 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 15:00:45,323 INFO [Listener at localhost/36191] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 15:00:45,324 INFO [Listener at localhost/36191] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37217 2023-02-11 15:00:45,325 INFO [Listener at localhost/36191] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 15:00:45,325 DEBUG [Listener at localhost/36191] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 15:00:45,326 INFO [Listener at localhost/36191] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:45,328 INFO [Listener at localhost/36191] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:45,330 INFO [Listener at localhost/36191] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37217 connecting to ZooKeeper ensemble=127.0.0.1:63275 2023-02-11 15:00:45,334 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:372170x0, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 15:00:45,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:37217-0x10043e6106d0002 connected 2023-02-11 15:00:45,335 DEBUG [Listener at localhost/36191] zookeeper.ZKUtil(164): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 15:00:45,336 DEBUG [Listener at localhost/36191] zookeeper.ZKUtil(164): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 15:00:45,336 DEBUG [Listener at localhost/36191] zookeeper.ZKUtil(164): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 15:00:45,337 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37217 2023-02-11 15:00:45,337 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37217 2023-02-11 15:00:45,337 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37217 2023-02-11 15:00:45,338 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37217 2023-02-11 15:00:45,338 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37217 2023-02-11 15:00:45,351 INFO [Listener at localhost/36191] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-11 15:00:45,351 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:45,352 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:45,352 INFO [Listener at localhost/36191] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 15:00:45,352 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:45,352 INFO [Listener at localhost/36191] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 15:00:45,352 INFO [Listener at localhost/36191] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 15:00:45,353 INFO [Listener at localhost/36191] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46559 2023-02-11 15:00:45,354 INFO [Listener at localhost/36191] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 15:00:45,355 DEBUG [Listener at localhost/36191] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 15:00:45,356 INFO [Listener at localhost/36191] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:45,357 INFO [Listener at localhost/36191] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:45,358 INFO [Listener at localhost/36191] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46559 connecting to ZooKeeper ensemble=127.0.0.1:63275 2023-02-11 15:00:45,362 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:465590x0, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 15:00:45,363 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:46559-0x10043e6106d0003 connected 2023-02-11 15:00:45,363 DEBUG [Listener at localhost/36191] zookeeper.ZKUtil(164): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 15:00:45,363 DEBUG [Listener at localhost/36191] zookeeper.ZKUtil(164): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 15:00:45,364 DEBUG [Listener at localhost/36191] zookeeper.ZKUtil(164): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 15:00:45,365 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46559 2023-02-11 15:00:45,365 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46559 2023-02-11 15:00:45,365 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46559 2023-02-11 15:00:45,366 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46559 2023-02-11 15:00:45,366 DEBUG [Listener at localhost/36191] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46559 2023-02-11 15:00:45,368 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,37295,1676127644151 2023-02-11 15:00:45,380 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-11 15:00:45,381 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,37295,1676127644151 2023-02-11 15:00:45,399 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 15:00:45,399 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 15:00:45,399 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 15:00:45,399 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 15:00:45,401 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:45,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-11 15:00:45,403 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,37295,1676127644151 from backup master directory 2023-02-11 15:00:45,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-11 15:00:45,406 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,37295,1676127644151 2023-02-11 15:00:45,406 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-11 15:00:45,407 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 15:00:45,407 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,37295,1676127644151 2023-02-11 15:00:45,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-11 15:00:45,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-11 15:00:45,498 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/hbase.id with ID: c6074d92-3c46-46b2-bdbd-1df7c0d94000 2023-02-11 15:00:45,541 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:45,555 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:45,602 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x5c962d1a to 127.0.0.1:63275 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 15:00:45,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@61fca7f0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 15:00:45,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-11 15:00:45,660 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-11 15:00:45,677 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-11 15:00:45,677 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-11 15:00:45,678 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-11 15:00:45,682 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-11 15:00:45,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 15:00:45,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/MasterData/data/master/store-tmp 2023-02-11 15:00:45,783 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 15:00:45,783 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-11 15:00:45,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:45,784 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:45,784 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-11 15:00:45,784 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:45,784 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:45,784 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676127645783Disabling compacts and flushes for region at 1676127645783Disabling writes for close at 1676127645784 (+1 ms)Writing region close event to WAL at 1676127645784Closed at 1676127645784 2023-02-11 15:00:45,785 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/MasterData/WALs/jenkins-hbase4.apache.org,37295,1676127644151 2023-02-11 15:00:45,805 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37295%2C1676127644151, suffix=, logDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/MasterData/WALs/jenkins-hbase4.apache.org,37295,1676127644151, archiveDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/MasterData/oldWALs, maxLogs=10 2023-02-11 15:00:45,860 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42823,DS-c12805c7-d335-4d6f-b427-bcd385ac8afa,DISK] 2023-02-11 15:00:45,860 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40405,DS-ad5ccbfc-2075-4a9a-acab-951204632a80,DISK] 2023-02-11 15:00:45,860 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44679,DS-f39f2519-553b-44cd-943f-211e5b0b456a,DISK] 2023-02-11 15:00:45,868 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-11 15:00:45,937 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/MasterData/WALs/jenkins-hbase4.apache.org,37295,1676127644151/jenkins-hbase4.apache.org%2C37295%2C1676127644151.1676127645815 2023-02-11 15:00:45,938 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44679,DS-f39f2519-553b-44cd-943f-211e5b0b456a,DISK], DatanodeInfoWithStorage[127.0.0.1:42823,DS-c12805c7-d335-4d6f-b427-bcd385ac8afa,DISK], DatanodeInfoWithStorage[127.0.0.1:40405,DS-ad5ccbfc-2075-4a9a-acab-951204632a80,DISK]] 2023-02-11 15:00:45,938 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-11 15:00:45,939 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 15:00:45,942 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 15:00:45,943 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 15:00:45,999 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-11 15:00:46,005 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-11 15:00:46,034 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-11 15:00:46,045 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:46,050 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-11 15:00:46,052 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-11 15:00:46,065 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 15:00:46,069 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 15:00:46,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73623453, jitterRate=0.09707494080066681}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 15:00:46,070 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676127645969Initializing all the Stores at 1676127645971 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676127645971Cleaning up temporary data from old regions at 1676127646057 (+86 ms)Cleaning up detritus from prior splits at 1676127646058 (+1 ms)Region opened successfully at 1676127646070 (+12 ms) 2023-02-11 15:00:46,072 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-11 15:00:46,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-11 15:00:46,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-11 15:00:46,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 15:00:46,094 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 15:00:46,125 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 30 msec 2023-02-11 15:00:46,125 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-11 15:00:46,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-11 15:00:46,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-11 15:00:46,180 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-11 15:00:46,183 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-11 15:00:46,184 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-11 15:00:46,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-11 15:00:46,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-11 15:00:46,195 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:46,197 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-11 15:00:46,198 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-11 15:00:46,209 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-11 15:00:46,213 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 15:00:46,213 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 15:00:46,213 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 15:00:46,213 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 15:00:46,213 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:46,214 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,37295,1676127644151, sessionid=0x10043e6106d0000, setting cluster-up flag (Was=false) 2023-02-11 15:00:46,229 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:46,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-11 15:00:46,235 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37295,1676127644151 2023-02-11 15:00:46,241 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:46,246 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-11 15:00:46,248 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37295,1676127644151 2023-02-11 15:00:46,250 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/.hbase-snapshot/.tmp 2023-02-11 15:00:46,270 INFO [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer(952): ClusterId : c6074d92-3c46-46b2-bdbd-1df7c0d94000 2023-02-11 15:00:46,270 INFO [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer(952): ClusterId : c6074d92-3c46-46b2-bdbd-1df7c0d94000 2023-02-11 15:00:46,271 INFO [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(952): ClusterId : c6074d92-3c46-46b2-bdbd-1df7c0d94000 2023-02-11 15:00:46,276 DEBUG [RS:1;jenkins-hbase4:37217] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 15:00:46,276 DEBUG [RS:2;jenkins-hbase4:46559] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 15:00:46,276 DEBUG [RS:0;jenkins-hbase4:33713] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 15:00:46,282 DEBUG [RS:0;jenkins-hbase4:33713] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 15:00:46,282 DEBUG [RS:2;jenkins-hbase4:46559] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 15:00:46,282 DEBUG [RS:1;jenkins-hbase4:37217] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 15:00:46,282 DEBUG [RS:2;jenkins-hbase4:46559] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 15:00:46,282 DEBUG [RS:0;jenkins-hbase4:33713] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 15:00:46,282 DEBUG [RS:1;jenkins-hbase4:37217] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 15:00:46,286 DEBUG [RS:2;jenkins-hbase4:46559] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 15:00:46,286 DEBUG [RS:0;jenkins-hbase4:33713] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 15:00:46,286 DEBUG [RS:1;jenkins-hbase4:37217] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 15:00:46,289 DEBUG [RS:2;jenkins-hbase4:46559] zookeeper.ReadOnlyZKClient(139): Connect 0x0e97d46e to 127.0.0.1:63275 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 15:00:46,289 DEBUG [RS:0;jenkins-hbase4:33713] zookeeper.ReadOnlyZKClient(139): Connect 0x66ff83c1 to 127.0.0.1:63275 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 15:00:46,289 DEBUG [RS:1;jenkins-hbase4:37217] zookeeper.ReadOnlyZKClient(139): Connect 0x5d0f36d2 to 127.0.0.1:63275 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 15:00:46,296 DEBUG [RS:0;jenkins-hbase4:33713] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6838616e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 15:00:46,296 DEBUG [RS:2;jenkins-hbase4:46559] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@502bcc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 15:00:46,297 DEBUG [RS:0;jenkins-hbase4:33713] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c07e8b1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 15:00:46,297 DEBUG [RS:2;jenkins-hbase4:46559] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@46fbf1e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 15:00:46,298 DEBUG [RS:1;jenkins-hbase4:37217] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@53c8c303, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 15:00:46,298 DEBUG [RS:1;jenkins-hbase4:37217] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@57f75da2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 15:00:46,319 DEBUG [RS:0;jenkins-hbase4:33713] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:33713 2023-02-11 15:00:46,320 DEBUG [RS:1;jenkins-hbase4:37217] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:37217 2023-02-11 15:00:46,320 DEBUG [RS:2;jenkins-hbase4:46559] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:46559 2023-02-11 15:00:46,323 INFO [RS:1;jenkins-hbase4:37217] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 15:00:46,323 INFO [RS:2;jenkins-hbase4:46559] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 15:00:46,323 INFO [RS:1;jenkins-hbase4:37217] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 15:00:46,323 INFO [RS:0;jenkins-hbase4:33713] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 15:00:46,324 DEBUG [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 15:00:46,324 INFO [RS:2;jenkins-hbase4:46559] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 15:00:46,324 INFO [RS:0;jenkins-hbase4:33713] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 15:00:46,324 DEBUG [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 15:00:46,324 DEBUG [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 15:00:46,327 INFO [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37295,1676127644151 with isa=jenkins-hbase4.apache.org/172.31.14.131:37217, startcode=1676127645321 2023-02-11 15:00:46,327 INFO [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37295,1676127644151 with isa=jenkins-hbase4.apache.org/172.31.14.131:33713, startcode=1676127645277 2023-02-11 15:00:46,327 INFO [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37295,1676127644151 with isa=jenkins-hbase4.apache.org/172.31.14.131:46559, startcode=1676127645351 2023-02-11 15:00:46,343 DEBUG [RS:2;jenkins-hbase4:46559] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 15:00:46,343 DEBUG [RS:0;jenkins-hbase4:33713] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 15:00:46,343 DEBUG [RS:1;jenkins-hbase4:37217] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 15:00:46,396 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54917, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 15:00:46,396 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34513, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 15:00:46,396 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44055, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 15:00:46,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-11 15:00:46,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37295] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-11 15:00:46,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37295] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-11 15:00:46,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37295] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-11 15:00:46,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 15:00:46,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 15:00:46,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 15:00:46,413 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 15:00:46,413 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-11 15:00:46,413 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,413 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 15:00:46,413 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,415 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676127676415 2023-02-11 15:00:46,418 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-11 15:00:46,427 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-11 15:00:46,427 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-11 15:00:46,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-11 15:00:46,434 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-11 15:00:46,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-11 15:00:46,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-11 15:00:46,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-11 15:00:46,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-11 15:00:46,438 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-11 15:00:46,441 DEBUG [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer(2829): Master is not running yet 2023-02-11 15:00:46,441 DEBUG [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer(2829): Master is not running yet 2023-02-11 15:00:46,441 DEBUG [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(2829): Master is not running yet 2023-02-11 15:00:46,442 WARN [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-11 15:00:46,442 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-11 15:00:46,442 WARN [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-11 15:00:46,442 WARN [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-11 15:00:46,442 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-11 15:00:46,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-11 15:00:46,445 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-11 15:00:46,446 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676127646446,5,FailOnTimeoutGroup] 2023-02-11 15:00:46,446 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676127646446,5,FailOnTimeoutGroup] 2023-02-11 15:00:46,446 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,446 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-11 15:00:46,448 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,448 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,480 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-11 15:00:46,481 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-11 15:00:46,481 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46 2023-02-11 15:00:46,503 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 15:00:46,506 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-11 15:00:46,508 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/info 2023-02-11 15:00:46,509 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-11 15:00:46,510 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:46,510 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-11 15:00:46,512 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/rep_barrier 2023-02-11 15:00:46,513 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-11 15:00:46,514 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:46,514 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-11 15:00:46,516 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/table 2023-02-11 15:00:46,516 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-11 15:00:46,517 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:46,518 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740 2023-02-11 15:00:46,519 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740 2023-02-11 15:00:46,523 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-11 15:00:46,525 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-11 15:00:46,528 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 15:00:46,529 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74277524, jitterRate=0.10682135820388794}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-11 15:00:46,529 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676127646504Initializing all the Stores at 1676127646505 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676127646505Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676127646505Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676127646505Cleaning up temporary data from old regions at 1676127646521 (+16 ms)Cleaning up detritus from prior splits at 1676127646521Region opened successfully at 1676127646529 (+8 ms) 2023-02-11 15:00:46,529 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-11 15:00:46,529 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-11 15:00:46,530 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-11 15:00:46,530 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-11 15:00:46,530 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-11 15:00:46,531 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-11 15:00:46,531 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676127646529Disabling compacts and flushes for region at 1676127646529Disabling writes for close at 1676127646530 (+1 ms)Writing region close event to WAL at 1676127646531 (+1 ms)Closed at 1676127646531 2023-02-11 15:00:46,537 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-11 15:00:46,537 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-11 15:00:46,543 INFO [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37295,1676127644151 with isa=jenkins-hbase4.apache.org/172.31.14.131:37217, startcode=1676127645321 2023-02-11 15:00:46,543 INFO [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37295,1676127644151 with isa=jenkins-hbase4.apache.org/172.31.14.131:33713, startcode=1676127645277 2023-02-11 15:00:46,543 INFO [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37295,1676127644151 with isa=jenkins-hbase4.apache.org/172.31.14.131:46559, startcode=1676127645351 2023-02-11 15:00:46,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-11 15:00:46,550 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37295] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,37217,1676127645321 2023-02-11 15:00:46,550 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37295] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33713,1676127645277 2023-02-11 15:00:46,551 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37295] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46559,1676127645351 2023-02-11 15:00:46,558 DEBUG [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46 2023-02-11 15:00:46,558 DEBUG [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46 2023-02-11 15:00:46,558 DEBUG [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46 2023-02-11 15:00:46,558 DEBUG [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38847 2023-02-11 15:00:46,558 DEBUG [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38847 2023-02-11 15:00:46,558 DEBUG [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38847 2023-02-11 15:00:46,559 DEBUG [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 15:00:46,559 DEBUG [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 15:00:46,559 DEBUG [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 15:00:46,565 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:46,567 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-11 15:00:46,571 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-11 15:00:46,572 DEBUG [RS:1;jenkins-hbase4:37217] zookeeper.ZKUtil(162): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37217,1676127645321 2023-02-11 15:00:46,572 WARN [RS:1;jenkins-hbase4:37217] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 15:00:46,572 INFO [RS:1;jenkins-hbase4:37217] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 15:00:46,572 DEBUG [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/WALs/jenkins-hbase4.apache.org,37217,1676127645321 2023-02-11 15:00:46,572 DEBUG [RS:0;jenkins-hbase4:33713] zookeeper.ZKUtil(162): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33713,1676127645277 2023-02-11 15:00:46,572 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37217,1676127645321] 2023-02-11 15:00:46,572 WARN [RS:0;jenkins-hbase4:33713] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 15:00:46,573 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46559,1676127645351] 2023-02-11 15:00:46,573 INFO [RS:0;jenkins-hbase4:33713] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 15:00:46,573 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33713,1676127645277] 2023-02-11 15:00:46,574 DEBUG [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/WALs/jenkins-hbase4.apache.org,33713,1676127645277 2023-02-11 15:00:46,574 DEBUG [RS:2;jenkins-hbase4:46559] zookeeper.ZKUtil(162): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46559,1676127645351 2023-02-11 15:00:46,574 WARN [RS:2;jenkins-hbase4:46559] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 15:00:46,574 INFO [RS:2;jenkins-hbase4:46559] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 15:00:46,574 DEBUG [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/WALs/jenkins-hbase4.apache.org,46559,1676127645351 2023-02-11 15:00:46,586 DEBUG [RS:0;jenkins-hbase4:33713] zookeeper.ZKUtil(162): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33713,1676127645277 2023-02-11 15:00:46,586 DEBUG [RS:1;jenkins-hbase4:37217] zookeeper.ZKUtil(162): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33713,1676127645277 2023-02-11 15:00:46,586 DEBUG [RS:2;jenkins-hbase4:46559] zookeeper.ZKUtil(162): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33713,1676127645277 2023-02-11 15:00:46,587 DEBUG [RS:0;jenkins-hbase4:33713] zookeeper.ZKUtil(162): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37217,1676127645321 2023-02-11 15:00:46,587 DEBUG [RS:1;jenkins-hbase4:37217] zookeeper.ZKUtil(162): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37217,1676127645321 2023-02-11 15:00:46,587 DEBUG [RS:2;jenkins-hbase4:46559] zookeeper.ZKUtil(162): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37217,1676127645321 2023-02-11 15:00:46,587 DEBUG [RS:1;jenkins-hbase4:37217] zookeeper.ZKUtil(162): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46559,1676127645351 2023-02-11 15:00:46,587 DEBUG [RS:0;jenkins-hbase4:33713] zookeeper.ZKUtil(162): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46559,1676127645351 2023-02-11 15:00:46,587 DEBUG [RS:2;jenkins-hbase4:46559] zookeeper.ZKUtil(162): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46559,1676127645351 2023-02-11 15:00:46,597 DEBUG [RS:0;jenkins-hbase4:33713] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 15:00:46,597 DEBUG [RS:2;jenkins-hbase4:46559] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 15:00:46,597 DEBUG [RS:1;jenkins-hbase4:37217] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 15:00:46,606 INFO [RS:1;jenkins-hbase4:37217] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 15:00:46,606 INFO [RS:2;jenkins-hbase4:46559] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 15:00:46,606 INFO [RS:0;jenkins-hbase4:33713] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 15:00:46,625 INFO [RS:1;jenkins-hbase4:37217] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 15:00:46,625 INFO [RS:0;jenkins-hbase4:33713] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 15:00:46,625 INFO [RS:2;jenkins-hbase4:46559] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 15:00:46,630 INFO [RS:2;jenkins-hbase4:46559] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 15:00:46,630 INFO [RS:1;jenkins-hbase4:37217] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 15:00:46,630 INFO [RS:2;jenkins-hbase4:46559] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,630 INFO [RS:0;jenkins-hbase4:33713] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 15:00:46,631 INFO [RS:1;jenkins-hbase4:37217] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,631 INFO [RS:0;jenkins-hbase4:33713] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,631 INFO [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 15:00:46,632 INFO [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 15:00:46,631 INFO [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 15:00:46,639 INFO [RS:2;jenkins-hbase4:46559] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,639 INFO [RS:1;jenkins-hbase4:37217] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,639 INFO [RS:0;jenkins-hbase4:33713] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,639 DEBUG [RS:2;jenkins-hbase4:46559] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,640 DEBUG [RS:1;jenkins-hbase4:37217] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,640 DEBUG [RS:2;jenkins-hbase4:46559] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,640 DEBUG [RS:0;jenkins-hbase4:33713] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,640 DEBUG [RS:2;jenkins-hbase4:46559] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,640 DEBUG [RS:1;jenkins-hbase4:37217] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,640 DEBUG [RS:2;jenkins-hbase4:46559] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,640 DEBUG [RS:1;jenkins-hbase4:37217] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,641 DEBUG [RS:2;jenkins-hbase4:46559] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,641 DEBUG [RS:1;jenkins-hbase4:37217] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,641 DEBUG [RS:2;jenkins-hbase4:46559] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 15:00:46,641 DEBUG [RS:1;jenkins-hbase4:37217] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,641 DEBUG [RS:2;jenkins-hbase4:46559] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,641 DEBUG [RS:1;jenkins-hbase4:37217] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 15:00:46,640 DEBUG [RS:0;jenkins-hbase4:33713] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,641 DEBUG [RS:1;jenkins-hbase4:37217] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,641 DEBUG [RS:2;jenkins-hbase4:46559] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,641 DEBUG [RS:1;jenkins-hbase4:37217] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,641 DEBUG [RS:2;jenkins-hbase4:46559] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,641 DEBUG [RS:1;jenkins-hbase4:37217] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,641 DEBUG [RS:2;jenkins-hbase4:46559] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,641 DEBUG [RS:1;jenkins-hbase4:37217] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,641 DEBUG [RS:0;jenkins-hbase4:33713] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,641 DEBUG [RS:0;jenkins-hbase4:33713] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,642 DEBUG [RS:0;jenkins-hbase4:33713] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,642 DEBUG [RS:0;jenkins-hbase4:33713] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 15:00:46,643 DEBUG [RS:0;jenkins-hbase4:33713] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,643 INFO [RS:1;jenkins-hbase4:37217] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,643 DEBUG [RS:0;jenkins-hbase4:33713] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,643 INFO [RS:1;jenkins-hbase4:37217] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,643 DEBUG [RS:0;jenkins-hbase4:33713] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,643 INFO [RS:1;jenkins-hbase4:37217] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,643 DEBUG [RS:0;jenkins-hbase4:33713] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:46,645 INFO [RS:2;jenkins-hbase4:46559] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,645 INFO [RS:2;jenkins-hbase4:46559] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,645 INFO [RS:2;jenkins-hbase4:46559] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,648 INFO [RS:0;jenkins-hbase4:33713] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,648 INFO [RS:0;jenkins-hbase4:33713] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,648 INFO [RS:0;jenkins-hbase4:33713] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,665 INFO [RS:0;jenkins-hbase4:33713] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 15:00:46,665 INFO [RS:2;jenkins-hbase4:46559] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 15:00:46,665 INFO [RS:1;jenkins-hbase4:37217] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 15:00:46,669 INFO [RS:2;jenkins-hbase4:46559] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46559,1676127645351-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,669 INFO [RS:0;jenkins-hbase4:33713] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33713,1676127645277-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,669 INFO [RS:1;jenkins-hbase4:37217] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37217,1676127645321-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:46,692 INFO [RS:0;jenkins-hbase4:33713] regionserver.Replication(203): jenkins-hbase4.apache.org,33713,1676127645277 started 2023-02-11 15:00:46,692 INFO [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,33713,1676127645277, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33713, sessionid=0x10043e6106d0001 2023-02-11 15:00:46,693 DEBUG [RS:0;jenkins-hbase4:33713] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 15:00:46,693 DEBUG [RS:0;jenkins-hbase4:33713] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33713,1676127645277 2023-02-11 15:00:46,693 DEBUG [RS:0;jenkins-hbase4:33713] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33713,1676127645277' 2023-02-11 15:00:46,693 DEBUG [RS:0;jenkins-hbase4:33713] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 15:00:46,694 DEBUG [RS:0;jenkins-hbase4:33713] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 15:00:46,694 DEBUG [RS:0;jenkins-hbase4:33713] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 15:00:46,694 DEBUG [RS:0;jenkins-hbase4:33713] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 15:00:46,694 DEBUG [RS:0;jenkins-hbase4:33713] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33713,1676127645277 2023-02-11 15:00:46,694 DEBUG [RS:0;jenkins-hbase4:33713] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33713,1676127645277' 2023-02-11 15:00:46,695 DEBUG [RS:0;jenkins-hbase4:33713] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 15:00:46,695 DEBUG [RS:0;jenkins-hbase4:33713] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 15:00:46,695 INFO [RS:2;jenkins-hbase4:46559] regionserver.Replication(203): jenkins-hbase4.apache.org,46559,1676127645351 started 2023-02-11 15:00:46,696 INFO [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,46559,1676127645351, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46559, sessionid=0x10043e6106d0003 2023-02-11 15:00:46,696 DEBUG [RS:0;jenkins-hbase4:33713] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 15:00:46,696 DEBUG [RS:2;jenkins-hbase4:46559] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 15:00:46,696 INFO [RS:0;jenkins-hbase4:33713] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 15:00:46,696 DEBUG [RS:2;jenkins-hbase4:46559] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46559,1676127645351 2023-02-11 15:00:46,701 INFO [RS:0;jenkins-hbase4:33713] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 15:00:46,701 DEBUG [RS:2;jenkins-hbase4:46559] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46559,1676127645351' 2023-02-11 15:00:46,701 DEBUG [RS:2;jenkins-hbase4:46559] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 15:00:46,702 DEBUG [RS:2;jenkins-hbase4:46559] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 15:00:46,703 DEBUG [RS:2;jenkins-hbase4:46559] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 15:00:46,703 DEBUG [RS:2;jenkins-hbase4:46559] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 15:00:46,703 DEBUG [RS:2;jenkins-hbase4:46559] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46559,1676127645351 2023-02-11 15:00:46,703 DEBUG [RS:2;jenkins-hbase4:46559] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46559,1676127645351' 2023-02-11 15:00:46,704 DEBUG [RS:2;jenkins-hbase4:46559] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 15:00:46,704 DEBUG [RS:2;jenkins-hbase4:46559] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 15:00:46,705 DEBUG [RS:2;jenkins-hbase4:46559] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 15:00:46,705 INFO [RS:2;jenkins-hbase4:46559] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 15:00:46,705 INFO [RS:2;jenkins-hbase4:46559] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 15:00:46,707 INFO [RS:1;jenkins-hbase4:37217] regionserver.Replication(203): jenkins-hbase4.apache.org,37217,1676127645321 started 2023-02-11 15:00:46,707 INFO [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,37217,1676127645321, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37217, sessionid=0x10043e6106d0002 2023-02-11 15:00:46,707 DEBUG [RS:1;jenkins-hbase4:37217] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 15:00:46,707 DEBUG [RS:1;jenkins-hbase4:37217] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37217,1676127645321 2023-02-11 15:00:46,707 DEBUG [RS:1;jenkins-hbase4:37217] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37217,1676127645321' 2023-02-11 15:00:46,707 DEBUG [RS:1;jenkins-hbase4:37217] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 15:00:46,708 DEBUG [RS:1;jenkins-hbase4:37217] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 15:00:46,708 DEBUG [RS:1;jenkins-hbase4:37217] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 15:00:46,708 DEBUG [RS:1;jenkins-hbase4:37217] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 15:00:46,708 DEBUG [RS:1;jenkins-hbase4:37217] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37217,1676127645321 2023-02-11 15:00:46,708 DEBUG [RS:1;jenkins-hbase4:37217] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37217,1676127645321' 2023-02-11 15:00:46,708 DEBUG [RS:1;jenkins-hbase4:37217] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 15:00:46,709 DEBUG [RS:1;jenkins-hbase4:37217] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 15:00:46,709 DEBUG [RS:1;jenkins-hbase4:37217] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 15:00:46,709 INFO [RS:1;jenkins-hbase4:37217] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 15:00:46,709 INFO [RS:1;jenkins-hbase4:37217] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 15:00:46,723 DEBUG [jenkins-hbase4:37295] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-11 15:00:46,727 DEBUG [jenkins-hbase4:37295] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 15:00:46,732 DEBUG [jenkins-hbase4:37295] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 15:00:46,732 DEBUG [jenkins-hbase4:37295] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 15:00:46,732 DEBUG [jenkins-hbase4:37295] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 15:00:46,732 DEBUG [jenkins-hbase4:37295] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 15:00:46,735 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33713,1676127645277, state=OPENING 2023-02-11 15:00:46,742 DEBUG [PEWorker-5] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-11 15:00:46,743 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:46,743 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-11 15:00:46,747 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33713,1676127645277}] 2023-02-11 15:00:46,812 INFO [RS:1;jenkins-hbase4:37217] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37217%2C1676127645321, suffix=, logDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/WALs/jenkins-hbase4.apache.org,37217,1676127645321, archiveDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/oldWALs, maxLogs=32 2023-02-11 15:00:46,812 INFO [RS:0;jenkins-hbase4:33713] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33713%2C1676127645277, suffix=, logDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/WALs/jenkins-hbase4.apache.org,33713,1676127645277, archiveDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/oldWALs, maxLogs=32 2023-02-11 15:00:46,812 INFO [RS:2;jenkins-hbase4:46559] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46559%2C1676127645351, suffix=, logDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/WALs/jenkins-hbase4.apache.org,46559,1676127645351, archiveDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/oldWALs, maxLogs=32 2023-02-11 15:00:46,835 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42823,DS-c12805c7-d335-4d6f-b427-bcd385ac8afa,DISK] 2023-02-11 15:00:46,835 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44679,DS-f39f2519-553b-44cd-943f-211e5b0b456a,DISK] 2023-02-11 15:00:46,835 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40405,DS-ad5ccbfc-2075-4a9a-acab-951204632a80,DISK] 2023-02-11 15:00:46,847 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44679,DS-f39f2519-553b-44cd-943f-211e5b0b456a,DISK] 2023-02-11 15:00:46,847 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40405,DS-ad5ccbfc-2075-4a9a-acab-951204632a80,DISK] 2023-02-11 15:00:46,847 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42823,DS-c12805c7-d335-4d6f-b427-bcd385ac8afa,DISK] 2023-02-11 15:00:46,850 INFO [RS:2;jenkins-hbase4:46559] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/WALs/jenkins-hbase4.apache.org,46559,1676127645351/jenkins-hbase4.apache.org%2C46559%2C1676127645351.1676127646816 2023-02-11 15:00:46,851 DEBUG [RS:2;jenkins-hbase4:46559] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42823,DS-c12805c7-d335-4d6f-b427-bcd385ac8afa,DISK], DatanodeInfoWithStorage[127.0.0.1:40405,DS-ad5ccbfc-2075-4a9a-acab-951204632a80,DISK], DatanodeInfoWithStorage[127.0.0.1:44679,DS-f39f2519-553b-44cd-943f-211e5b0b456a,DISK]] 2023-02-11 15:00:46,857 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42823,DS-c12805c7-d335-4d6f-b427-bcd385ac8afa,DISK] 2023-02-11 15:00:46,857 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40405,DS-ad5ccbfc-2075-4a9a-acab-951204632a80,DISK] 2023-02-11 15:00:46,857 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44679,DS-f39f2519-553b-44cd-943f-211e5b0b456a,DISK] 2023-02-11 15:00:46,862 INFO [RS:0;jenkins-hbase4:33713] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/WALs/jenkins-hbase4.apache.org,33713,1676127645277/jenkins-hbase4.apache.org%2C33713%2C1676127645277.1676127646816 2023-02-11 15:00:46,863 DEBUG [RS:0;jenkins-hbase4:33713] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42823,DS-c12805c7-d335-4d6f-b427-bcd385ac8afa,DISK], DatanodeInfoWithStorage[127.0.0.1:40405,DS-ad5ccbfc-2075-4a9a-acab-951204632a80,DISK], DatanodeInfoWithStorage[127.0.0.1:44679,DS-f39f2519-553b-44cd-943f-211e5b0b456a,DISK]] 2023-02-11 15:00:46,863 INFO [RS:1;jenkins-hbase4:37217] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/WALs/jenkins-hbase4.apache.org,37217,1676127645321/jenkins-hbase4.apache.org%2C37217%2C1676127645321.1676127646816 2023-02-11 15:00:46,863 DEBUG [RS:1;jenkins-hbase4:37217] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40405,DS-ad5ccbfc-2075-4a9a-acab-951204632a80,DISK], DatanodeInfoWithStorage[127.0.0.1:42823,DS-c12805c7-d335-4d6f-b427-bcd385ac8afa,DISK], DatanodeInfoWithStorage[127.0.0.1:44679,DS-f39f2519-553b-44cd-943f-211e5b0b456a,DISK]] 2023-02-11 15:00:46,931 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33713,1676127645277 2023-02-11 15:00:46,933 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 15:00:46,936 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53890, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 15:00:46,949 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-11 15:00:46,949 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 15:00:46,953 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33713%2C1676127645277.meta, suffix=.meta, logDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/WALs/jenkins-hbase4.apache.org,33713,1676127645277, archiveDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/oldWALs, maxLogs=32 2023-02-11 15:00:46,973 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40405,DS-ad5ccbfc-2075-4a9a-acab-951204632a80,DISK] 2023-02-11 15:00:46,974 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44679,DS-f39f2519-553b-44cd-943f-211e5b0b456a,DISK] 2023-02-11 15:00:46,974 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42823,DS-c12805c7-d335-4d6f-b427-bcd385ac8afa,DISK] 2023-02-11 15:00:46,980 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/WALs/jenkins-hbase4.apache.org,33713,1676127645277/jenkins-hbase4.apache.org%2C33713%2C1676127645277.meta.1676127646954.meta 2023-02-11 15:00:46,981 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40405,DS-ad5ccbfc-2075-4a9a-acab-951204632a80,DISK], DatanodeInfoWithStorage[127.0.0.1:44679,DS-f39f2519-553b-44cd-943f-211e5b0b456a,DISK], DatanodeInfoWithStorage[127.0.0.1:42823,DS-c12805c7-d335-4d6f-b427-bcd385ac8afa,DISK]] 2023-02-11 15:00:46,981 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-11 15:00:46,985 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-11 15:00:47,002 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-11 15:00:47,007 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-11 15:00:47,012 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-11 15:00:47,012 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 15:00:47,012 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-11 15:00:47,012 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-11 15:00:47,015 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-11 15:00:47,017 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/info 2023-02-11 15:00:47,017 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/info 2023-02-11 15:00:47,018 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-11 15:00:47,018 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:47,019 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-11 15:00:47,020 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/rep_barrier 2023-02-11 15:00:47,020 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/rep_barrier 2023-02-11 15:00:47,021 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-11 15:00:47,021 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:47,021 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-11 15:00:47,023 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/table 2023-02-11 15:00:47,023 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/table 2023-02-11 15:00:47,023 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-11 15:00:47,024 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:47,026 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740 2023-02-11 15:00:47,028 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740 2023-02-11 15:00:47,031 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-11 15:00:47,033 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-11 15:00:47,035 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67121659, jitterRate=1.9066035747528076E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-11 15:00:47,035 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676127647013Writing region info on filesystem at 1676127647013Initializing all the Stores at 1676127647015 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676127647015Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676127647015Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676127647015Cleaning up temporary data from old regions at 1676127647030 (+15 ms)Cleaning up detritus from prior splits at 1676127647030Running coprocessor post-open hooks at 1676127647035 (+5 ms)Region opened successfully at 1676127647035 2023-02-11 15:00:47,046 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676127646923 2023-02-11 15:00:47,063 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-11 15:00:47,063 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-11 15:00:47,064 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33713,1676127645277, state=OPEN 2023-02-11 15:00:47,067 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-11 15:00:47,067 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-11 15:00:47,072 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-11 15:00:47,072 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33713,1676127645277 in 320 msec 2023-02-11 15:00:47,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-11 15:00:47,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 525 msec 2023-02-11 15:00:47,082 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 772 msec 2023-02-11 15:00:47,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676127645396, completionTime=-1 2023-02-11 15:00:47,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-11 15:00:47,082 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 15:00:47,148 DEBUG [hconnection-0x5d50ff84-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 15:00:47,151 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53902, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 15:00:47,167 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 15:00:47,168 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676127707168 2023-02-11 15:00:47,168 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676127767168 2023-02-11 15:00:47,168 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 85 msec 2023-02-11 15:00:47,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37295,1676127644151-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:47,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37295,1676127644151-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:47,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37295,1676127644151-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:47,198 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37295, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:47,198 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:47,204 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-11 15:00:47,214 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-11 15:00:47,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-11 15:00:47,224 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-11 15:00:47,227 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 15:00:47,229 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 15:00:47,251 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/.tmp/data/hbase/namespace/a8870a83df638bea15b8279ca1c5eb14 2023-02-11 15:00:47,253 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/.tmp/data/hbase/namespace/a8870a83df638bea15b8279ca1c5eb14 empty. 2023-02-11 15:00:47,254 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/.tmp/data/hbase/namespace/a8870a83df638bea15b8279ca1c5eb14 2023-02-11 15:00:47,254 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-11 15:00:47,291 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-11 15:00:47,294 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => a8870a83df638bea15b8279ca1c5eb14, NAME => 'hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/.tmp 2023-02-11 15:00:47,317 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 15:00:47,318 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing a8870a83df638bea15b8279ca1c5eb14, disabling compactions & flushes 2023-02-11 15:00:47,318 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. 2023-02-11 15:00:47,318 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. 2023-02-11 15:00:47,318 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. after waiting 0 ms 2023-02-11 15:00:47,318 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. 2023-02-11 15:00:47,318 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. 2023-02-11 15:00:47,318 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for a8870a83df638bea15b8279ca1c5eb14: Waiting for close lock at 1676127647318Disabling compacts and flushes for region at 1676127647318Disabling writes for close at 1676127647318Writing region close event to WAL at 1676127647318Closed at 1676127647318 2023-02-11 15:00:47,322 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 15:00:47,337 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676127647325"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676127647325"}]},"ts":"1676127647325"} 2023-02-11 15:00:47,364 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 15:00:47,365 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 15:00:47,370 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676127647366"}]},"ts":"1676127647366"} 2023-02-11 15:00:47,377 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-11 15:00:47,381 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 15:00:47,382 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 15:00:47,382 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 15:00:47,382 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 15:00:47,382 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 15:00:47,385 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=a8870a83df638bea15b8279ca1c5eb14, ASSIGN}] 2023-02-11 15:00:47,388 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=a8870a83df638bea15b8279ca1c5eb14, ASSIGN 2023-02-11 15:00:47,390 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=a8870a83df638bea15b8279ca1c5eb14, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33713,1676127645277; forceNewPlan=false, retain=false 2023-02-11 15:00:47,543 INFO [jenkins-hbase4:37295] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 15:00:47,543 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=a8870a83df638bea15b8279ca1c5eb14, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33713,1676127645277 2023-02-11 15:00:47,544 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676127647543"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676127647543"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676127647543"}]},"ts":"1676127647543"} 2023-02-11 15:00:47,548 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure a8870a83df638bea15b8279ca1c5eb14, server=jenkins-hbase4.apache.org,33713,1676127645277}] 2023-02-11 15:00:47,713 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. 2023-02-11 15:00:47,713 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a8870a83df638bea15b8279ca1c5eb14, NAME => 'hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14.', STARTKEY => '', ENDKEY => ''} 2023-02-11 15:00:47,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace a8870a83df638bea15b8279ca1c5eb14 2023-02-11 15:00:47,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 15:00:47,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a8870a83df638bea15b8279ca1c5eb14 2023-02-11 15:00:47,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a8870a83df638bea15b8279ca1c5eb14 2023-02-11 15:00:47,716 INFO [StoreOpener-a8870a83df638bea15b8279ca1c5eb14-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region a8870a83df638bea15b8279ca1c5eb14 2023-02-11 15:00:47,718 DEBUG [StoreOpener-a8870a83df638bea15b8279ca1c5eb14-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/namespace/a8870a83df638bea15b8279ca1c5eb14/info 2023-02-11 15:00:47,718 DEBUG [StoreOpener-a8870a83df638bea15b8279ca1c5eb14-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/namespace/a8870a83df638bea15b8279ca1c5eb14/info 2023-02-11 15:00:47,719 INFO [StoreOpener-a8870a83df638bea15b8279ca1c5eb14-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a8870a83df638bea15b8279ca1c5eb14 columnFamilyName info 2023-02-11 15:00:47,720 INFO [StoreOpener-a8870a83df638bea15b8279ca1c5eb14-1] regionserver.HStore(310): Store=a8870a83df638bea15b8279ca1c5eb14/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:47,721 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/namespace/a8870a83df638bea15b8279ca1c5eb14 2023-02-11 15:00:47,722 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/namespace/a8870a83df638bea15b8279ca1c5eb14 2023-02-11 15:00:47,727 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a8870a83df638bea15b8279ca1c5eb14 2023-02-11 15:00:47,731 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/namespace/a8870a83df638bea15b8279ca1c5eb14/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 15:00:47,732 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a8870a83df638bea15b8279ca1c5eb14; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64617363, jitterRate=-0.03712625801563263}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 15:00:47,732 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a8870a83df638bea15b8279ca1c5eb14: Running coprocessor pre-open hook at 1676127647714Writing region info on filesystem at 1676127647714Initializing all the Stores at 1676127647716 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676127647716Cleaning up temporary data from old regions at 1676127647723 (+7 ms)Cleaning up detritus from prior splits at 1676127647723Running coprocessor post-open hooks at 1676127647732 (+9 ms)Region opened successfully at 1676127647732 2023-02-11 15:00:47,734 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14., pid=6, masterSystemTime=1676127647702 2023-02-11 15:00:47,737 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. 2023-02-11 15:00:47,737 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. 2023-02-11 15:00:47,739 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=a8870a83df638bea15b8279ca1c5eb14, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33713,1676127645277 2023-02-11 15:00:47,739 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676127647738"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676127647738"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676127647738"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676127647738"}]},"ts":"1676127647738"} 2023-02-11 15:00:47,746 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-11 15:00:47,746 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure a8870a83df638bea15b8279ca1c5eb14, server=jenkins-hbase4.apache.org,33713,1676127645277 in 194 msec 2023-02-11 15:00:47,749 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-11 15:00:47,750 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=a8870a83df638bea15b8279ca1c5eb14, ASSIGN in 361 msec 2023-02-11 15:00:47,751 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 15:00:47,751 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676127647751"}]},"ts":"1676127647751"} 2023-02-11 15:00:47,753 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-11 15:00:47,758 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 15:00:47,760 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 542 msec 2023-02-11 15:00:47,827 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-11 15:00:47,828 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-11 15:00:47,828 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:47,860 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-11 15:00:47,876 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 15:00:47,882 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 29 msec 2023-02-11 15:00:47,894 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-11 15:00:47,905 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 15:00:47,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-02-11 15:00:47,919 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-11 15:00:47,922 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-11 15:00:47,922 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.515sec 2023-02-11 15:00:47,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-11 15:00:47,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-11 15:00:47,927 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-11 15:00:47,928 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37295,1676127644151-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-11 15:00:47,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37295,1676127644151-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-11 15:00:47,938 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-11 15:00:47,977 DEBUG [Listener at localhost/36191] zookeeper.ReadOnlyZKClient(139): Connect 0x13ca348a to 127.0.0.1:63275 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 15:00:47,982 DEBUG [Listener at localhost/36191] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@772c5efc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 15:00:47,996 DEBUG [hconnection-0x317fb24e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 15:00:48,039 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53916, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 15:00:48,049 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,37295,1676127644151 2023-02-11 15:00:48,049 DEBUG [Listener at localhost/36191] zookeeper.ReadOnlyZKClient(139): Connect 0x06120a40 to 127.0.0.1:63275 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 15:00:48,060 DEBUG [ReadOnlyZKClient-127.0.0.1:63275@0x06120a40] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@213ec4be, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 15:00:48,095 DEBUG [Listener at localhost/36191] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 15:00:48,099 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53922, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 15:00:48,100 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33713] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33713,1676127645277' ***** 2023-02-11 15:00:48,100 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33713] regionserver.HRegionServer(2310): STOPPED: Called by admin client org.apache.hadoop.hbase.client.AsyncConnectionImpl@24276c0 2023-02-11 15:00:48,100 INFO [RS:0;jenkins-hbase4:33713] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 15:00:48,101 INFO [RS:0;jenkins-hbase4:33713] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 15:00:48,101 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 15:00:48,102 INFO [RS:0;jenkins-hbase4:33713] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 15:00:48,102 INFO [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(3304): Received CLOSE for a8870a83df638bea15b8279ca1c5eb14 2023-02-11 15:00:48,103 INFO [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33713,1676127645277 2023-02-11 15:00:48,104 DEBUG [RS:0;jenkins-hbase4:33713] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x66ff83c1 to 127.0.0.1:63275 2023-02-11 15:00:48,104 DEBUG [RS:0;jenkins-hbase4:33713] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:48,104 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a8870a83df638bea15b8279ca1c5eb14, disabling compactions & flushes 2023-02-11 15:00:48,104 INFO [RS:0;jenkins-hbase4:33713] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 15:00:48,104 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. 2023-02-11 15:00:48,104 INFO [RS:0;jenkins-hbase4:33713] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 15:00:48,104 INFO [RS:0;jenkins-hbase4:33713] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 15:00:48,104 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. 2023-02-11 15:00:48,104 INFO [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-11 15:00:48,104 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. after waiting 0 ms 2023-02-11 15:00:48,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. 2023-02-11 15:00:48,105 INFO [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-02-11 15:00:48,105 DEBUG [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740, a8870a83df638bea15b8279ca1c5eb14=hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14.} 2023-02-11 15:00:48,106 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-11 15:00:48,106 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-11 15:00:48,106 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-11 15:00:48,106 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-11 15:00:48,106 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-11 15:00:48,106 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing a8870a83df638bea15b8279ca1c5eb14 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-11 15:00:48,106 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-02-11 15:00:48,106 DEBUG [Listener at localhost/36191] client.ConnectionUtils(586): Start fetching master stub from registry 2023-02-11 15:00:48,107 DEBUG [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(1505): Waiting on 1588230740, a8870a83df638bea15b8279ca1c5eb14 2023-02-11 15:00:48,111 DEBUG [ReadOnlyZKClient-127.0.0.1:63275@0x06120a40] client.AsyncConnectionImpl(289): The fetched master address is jenkins-hbase4.apache.org,37295,1676127644151 2023-02-11 15:00:48,114 DEBUG [ReadOnlyZKClient-127.0.0.1:63275@0x06120a40] client.ConnectionUtils(594): The fetched master stub is org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$Stub@78ffde27 2023-02-11 15:00:48,119 DEBUG [ReadOnlyZKClient-127.0.0.1:63275@0x06120a40] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 15:00:48,123 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38184, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 15:00:48,123 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37295] master.MasterRpcServices(1601): Client=jenkins//172.31.14.131 stop 2023-02-11 15:00:48,123 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37295] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37295,1676127644151' ***** 2023-02-11 15:00:48,123 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37295] regionserver.HRegionServer(2310): STOPPED: Stopped by RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37295 2023-02-11 15:00:48,124 DEBUG [M:0;jenkins-hbase4:37295] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6fbd3105, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 15:00:48,125 INFO [M:0;jenkins-hbase4:37295] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37295,1676127644151 2023-02-11 15:00:48,126 DEBUG [M:0;jenkins-hbase4:37295] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5c962d1a to 127.0.0.1:63275 2023-02-11 15:00:48,126 DEBUG [M:0;jenkins-hbase4:37295] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:48,127 INFO [M:0;jenkins-hbase4:37295] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37295,1676127644151; all regions closed. 2023-02-11 15:00:48,127 DEBUG [M:0;jenkins-hbase4:37295] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:48,127 DEBUG [M:0;jenkins-hbase4:37295] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-11 15:00:48,128 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-11 15:00:48,128 DEBUG [M:0;jenkins-hbase4:37295] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-11 15:00:48,128 INFO [M:0;jenkins-hbase4:37295] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-11 15:00:48,128 INFO [M:0;jenkins-hbase4:37295] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-11 15:00:48,128 INFO [M:0;jenkins-hbase4:37295] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-11 15:00:48,128 DEBUG [M:0;jenkins-hbase4:37295] master.HMaster(1502): Stopping service threads 2023-02-11 15:00:48,128 INFO [M:0;jenkins-hbase4:37295] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-11 15:00:48,129 INFO [M:0;jenkins-hbase4:37295] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 15:00:48,129 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-11 15:00:48,129 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-11 15:00:48,129 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676127646446] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676127646446,5,FailOnTimeoutGroup] 2023-02-11 15:00:48,129 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-11 15:00:48,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676127646446] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676127646446,5,FailOnTimeoutGroup] 2023-02-11 15:00:48,131 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-11 15:00:48,131 ERROR [M:0;jenkins-hbase4:37295] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] 2023-02-11 15:00:48,131 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:48,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 15:00:48,132 INFO [M:0;jenkins-hbase4:37295] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 15:00:48,132 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-11 15:00:48,132 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 15:00:48,132 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 15:00:48,133 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 15:00:48,133 DEBUG [M:0;jenkins-hbase4:37295] zookeeper.ZKUtil(398): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-11 15:00:48,133 WARN [M:0;jenkins-hbase4:37295] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-11 15:00:48,133 INFO [M:0;jenkins-hbase4:37295] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 15:00:48,133 INFO [M:0;jenkins-hbase4:37295] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-11 15:00:48,134 DEBUG [M:0;jenkins-hbase4:37295] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-11 15:00:48,134 INFO [M:0;jenkins-hbase4:37295] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:48,134 DEBUG [M:0;jenkins-hbase4:37295] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:48,134 DEBUG [M:0;jenkins-hbase4:37295] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-11 15:00:48,134 DEBUG [M:0;jenkins-hbase4:37295] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:48,135 INFO [M:0;jenkins-hbase4:37295] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-02-11 15:00:48,135 INFO [Listener at localhost/36191] client.AsyncConnectionImpl(207): Connection has been closed by Listener at localhost/36191. 2023-02-11 15:00:48,135 DEBUG [Listener at localhost/36191] client.AsyncConnectionImpl(232): Call stack: at java.lang.Thread.getStackTrace(Thread.java:1564) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:209) at org.apache.hbase.thirdparty.com.google.common.io.Closeables.close(Closeables.java:79) at org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2.tearDown(TestAsyncClusterAdminApi2.java:75) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2023-02-11 15:00:48,144 DEBUG [Listener at localhost/36191] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:48,144 DEBUG [Listener at localhost/36191] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x06120a40 to 127.0.0.1:63275 2023-02-11 15:00:48,145 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-11 15:00:48,147 DEBUG [Listener at localhost/36191] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x13ca348a to 127.0.0.1:63275 2023-02-11 15:00:48,147 DEBUG [Listener at localhost/36191] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:48,148 DEBUG [Listener at localhost/36191] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-11 15:00:48,149 INFO [Listener at localhost/36191] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37217,1676127645321' ***** 2023-02-11 15:00:48,149 INFO [Listener at localhost/36191] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 15:00:48,149 INFO [Listener at localhost/36191] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,46559,1676127645351' ***** 2023-02-11 15:00:48,149 INFO [Listener at localhost/36191] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 15:00:48,149 INFO [RS:1;jenkins-hbase4:37217] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 15:00:48,150 INFO [RS:2;jenkins-hbase4:46559] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 15:00:48,150 INFO [RS:1;jenkins-hbase4:37217] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 15:00:48,150 INFO [RS:2;jenkins-hbase4:46559] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 15:00:48,150 INFO [RS:1;jenkins-hbase4:37217] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 15:00:48,150 INFO [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37217,1676127645321 2023-02-11 15:00:48,150 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 15:00:48,150 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 15:00:48,150 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 15:00:48,151 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 15:00:48,150 DEBUG [RS:1;jenkins-hbase4:37217] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5d0f36d2 to 127.0.0.1:63275 2023-02-11 15:00:48,150 INFO [RS:2;jenkins-hbase4:46559] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 15:00:48,152 DEBUG [RS:1;jenkins-hbase4:37217] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:48,152 INFO [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,46559,1676127645351 2023-02-11 15:00:48,152 INFO [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37217,1676127645321; all regions closed. 2023-02-11 15:00:48,152 DEBUG [RS:2;jenkins-hbase4:46559] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e97d46e to 127.0.0.1:63275 2023-02-11 15:00:48,152 DEBUG [RS:2;jenkins-hbase4:46559] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:48,152 INFO [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,46559,1676127645351; all regions closed. 2023-02-11 15:00:48,173 DEBUG [RS:2;jenkins-hbase4:46559] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/oldWALs 2023-02-11 15:00:48,174 DEBUG [RS:1;jenkins-hbase4:37217] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/oldWALs 2023-02-11 15:00:48,174 INFO [RS:2;jenkins-hbase4:46559] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46559%2C1676127645351:(num 1676127646816) 2023-02-11 15:00:48,174 DEBUG [RS:2;jenkins-hbase4:46559] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:48,174 INFO [RS:1;jenkins-hbase4:37217] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37217%2C1676127645321:(num 1676127646816) 2023-02-11 15:00:48,174 INFO [RS:2;jenkins-hbase4:46559] regionserver.LeaseManager(133): Closed leases 2023-02-11 15:00:48,174 DEBUG [RS:1;jenkins-hbase4:37217] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:48,174 INFO [RS:1;jenkins-hbase4:37217] regionserver.LeaseManager(133): Closed leases 2023-02-11 15:00:48,174 INFO [RS:2;jenkins-hbase4:46559] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-11 15:00:48,174 INFO [RS:1;jenkins-hbase4:37217] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-11 15:00:48,175 INFO [RS:2;jenkins-hbase4:46559] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 15:00:48,175 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 15:00:48,175 INFO [RS:2;jenkins-hbase4:46559] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 15:00:48,175 INFO [RS:2;jenkins-hbase4:46559] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 15:00:48,175 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 15:00:48,175 INFO [RS:1;jenkins-hbase4:37217] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 15:00:48,175 INFO [RS:1;jenkins-hbase4:37217] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 15:00:48,175 INFO [RS:1;jenkins-hbase4:37217] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 15:00:48,176 INFO [RS:2;jenkins-hbase4:46559] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46559 2023-02-11 15:00:48,176 INFO [RS:1;jenkins-hbase4:37217] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37217 2023-02-11 15:00:48,216 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/.tmp/info/a878696fb16945298c485a51ddc2e7ff 2023-02-11 15:00:48,217 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/namespace/a8870a83df638bea15b8279ca1c5eb14/.tmp/info/b2c840ff84e04172879e4746d07a53ae 2023-02-11 15:00:48,219 INFO [M:0;jenkins-hbase4:37295] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a2dac53a2f8144a091daf1ccd48fe5ce 2023-02-11 15:00:48,246 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 15:00:48,257 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46559,1676127645351 2023-02-11 15:00:48,257 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:48,257 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46559,1676127645351 2023-02-11 15:00:48,257 ERROR [Listener at localhost/36191-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5c61eb4e rejected from java.util.concurrent.ThreadPoolExecutor@585e641a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:48,257 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:48,257 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46559,1676127645351 2023-02-11 15:00:48,258 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:48,258 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:48,258 ERROR [Listener at localhost/36191-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7f6360eb rejected from java.util.concurrent.ThreadPoolExecutor@585e641a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:48,258 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37217,1676127645321 2023-02-11 15:00:48,258 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37217,1676127645321 2023-02-11 15:00:48,258 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37217,1676127645321 2023-02-11 15:00:48,258 ERROR [Listener at localhost/36191-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7de00fd rejected from java.util.concurrent.ThreadPoolExecutor@585e641a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:48,259 ERROR [Listener at localhost/36191-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1402afe4 rejected from java.util.concurrent.ThreadPoolExecutor@103d39bc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:48,261 DEBUG [M:0;jenkins-hbase4:37295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a2dac53a2f8144a091daf1ccd48fe5ce as hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a2dac53a2f8144a091daf1ccd48fe5ce 2023-02-11 15:00:48,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/namespace/a8870a83df638bea15b8279ca1c5eb14/.tmp/info/b2c840ff84e04172879e4746d07a53ae as hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/namespace/a8870a83df638bea15b8279ca1c5eb14/info/b2c840ff84e04172879e4746d07a53ae 2023-02-11 15:00:48,273 INFO [M:0;jenkins-hbase4:37295] regionserver.HStore(1080): Added hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a2dac53a2f8144a091daf1ccd48fe5ce, entries=8, sequenceid=66, filesize=6.3 K 2023-02-11 15:00:48,274 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/namespace/a8870a83df638bea15b8279ca1c5eb14/info/b2c840ff84e04172879e4746d07a53ae, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 15:00:48,275 INFO [M:0;jenkins-hbase4:37295] regionserver.HRegion(2947): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 141ms, sequenceid=66, compaction requested=false 2023-02-11 15:00:48,280 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for a8870a83df638bea15b8279ca1c5eb14 in 174ms, sequenceid=6, compaction requested=false 2023-02-11 15:00:48,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-11 15:00:48,283 INFO [M:0;jenkins-hbase4:37295] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:48,284 DEBUG [M:0;jenkins-hbase4:37295] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676127648134Disabling compacts and flushes for region at 1676127648134Disabling writes for close at 1676127648134Obtaining lock to block concurrent updates at 1676127648135 (+1 ms)Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676127648135Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=24646, getHeapSize=30248, getOffHeapSize=0, getCellsCount=71 at 1676127648135Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676127648137 (+2 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676127648138 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676127648171 (+33 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676127648176 (+5 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676127648263 (+87 ms)Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 141ms, sequenceid=66, compaction requested=false at 1676127648275 (+12 ms)Writing region close event to WAL at 1676127648283 (+8 ms)Closed at 1676127648283 2023-02-11 15:00:48,293 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/.tmp/table/78ffcc453db34599bf871c842abef26c 2023-02-11 15:00:48,293 INFO [M:0;jenkins-hbase4:37295] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-11 15:00:48,294 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 15:00:48,294 INFO [M:0;jenkins-hbase4:37295] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37295 2023-02-11 15:00:48,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/namespace/a8870a83df638bea15b8279ca1c5eb14/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-11 15:00:48,301 DEBUG [M:0;jenkins-hbase4:37295] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,37295,1676127644151 already deleted, retry=false 2023-02-11 15:00:48,301 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. 2023-02-11 15:00:48,301 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a8870a83df638bea15b8279ca1c5eb14: Waiting for close lock at 1676127648104Running coprocessor pre-close hooks at 1676127648104Disabling compacts and flushes for region at 1676127648104Disabling writes for close at 1676127648105 (+1 ms)Obtaining lock to block concurrent updates at 1676127648106 (+1 ms)Preparing flush snapshotting stores in a8870a83df638bea15b8279ca1c5eb14 at 1676127648106Finished memstore snapshotting hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676127648116 (+10 ms)Flushing stores of hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. at 1676127648117 (+1 ms)Flushing a8870a83df638bea15b8279ca1c5eb14/info: creating writer at 1676127648121 (+4 ms)Flushing a8870a83df638bea15b8279ca1c5eb14/info: appending metadata at 1676127648170 (+49 ms)Flushing a8870a83df638bea15b8279ca1c5eb14/info: closing flushed file at 1676127648176 (+6 ms)Flushing a8870a83df638bea15b8279ca1c5eb14/info: reopening flushed file at 1676127648263 (+87 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for a8870a83df638bea15b8279ca1c5eb14 in 174ms, sequenceid=6, compaction requested=false at 1676127648280 (+17 ms)Writing region close event to WAL at 1676127648288 (+8 ms)Running coprocessor post-close hooks at 1676127648301 (+13 ms)Closed at 1676127648301 2023-02-11 15:00:48,302 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676127647214.a8870a83df638bea15b8279ca1c5eb14. 2023-02-11 15:00:48,307 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/.tmp/info/a878696fb16945298c485a51ddc2e7ff as hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/info/a878696fb16945298c485a51ddc2e7ff 2023-02-11 15:00:48,307 DEBUG [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-11 15:00:48,316 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/info/a878696fb16945298c485a51ddc2e7ff, entries=10, sequenceid=9, filesize=5.9 K 2023-02-11 15:00:48,317 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/.tmp/table/78ffcc453db34599bf871c842abef26c as hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/table/78ffcc453db34599bf871c842abef26c 2023-02-11 15:00:48,326 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/table/78ffcc453db34599bf871c842abef26c, entries=2, sequenceid=9, filesize=4.7 K 2023-02-11 15:00:48,328 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 222ms, sequenceid=9, compaction requested=false 2023-02-11 15:00:48,328 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-11 15:00:48,336 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-02-11 15:00:48,337 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-11 15:00:48,337 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-11 15:00:48,338 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676127648105Running coprocessor pre-close hooks at 1676127648105Disabling compacts and flushes for region at 1676127648105Disabling writes for close at 1676127648106 (+1 ms)Obtaining lock to block concurrent updates at 1676127648106Preparing flush snapshotting stores in 1588230740 at 1676127648106Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=1290, getHeapSize=2912, getOffHeapSize=0, getCellsCount=12 at 1676127648116 (+10 ms)Flushing stores of hbase:meta,,1.1588230740 at 1676127648117 (+1 ms)Flushing 1588230740/info: creating writer at 1676127648121 (+4 ms)Flushing 1588230740/info: appending metadata at 1676127648170 (+49 ms)Flushing 1588230740/info: closing flushed file at 1676127648176 (+6 ms)Flushing 1588230740/table: creating writer at 1676127648260 (+84 ms)Flushing 1588230740/table: appending metadata at 1676127648265 (+5 ms)Flushing 1588230740/table: closing flushed file at 1676127648265Flushing 1588230740/info: reopening flushed file at 1676127648308 (+43 ms)Flushing 1588230740/table: reopening flushed file at 1676127648318 (+10 ms)Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 222ms, sequenceid=9, compaction requested=false at 1676127648328 (+10 ms)Writing region close event to WAL at 1676127648331 (+3 ms)Running coprocessor post-close hooks at 1676127648337 (+6 ms)Closed at 1676127648337 2023-02-11 15:00:48,338 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-11 15:00:48,508 INFO [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33713,1676127645277; all regions closed. 2023-02-11 15:00:48,516 DEBUG [RS:0;jenkins-hbase4:33713] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/oldWALs 2023-02-11 15:00:48,516 INFO [RS:0;jenkins-hbase4:33713] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33713%2C1676127645277.meta:.meta(num 1676127646954) 2023-02-11 15:00:48,522 DEBUG [RS:0;jenkins-hbase4:33713] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/oldWALs 2023-02-11 15:00:48,522 INFO [RS:0;jenkins-hbase4:33713] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33713%2C1676127645277:(num 1676127646816) 2023-02-11 15:00:48,522 DEBUG [RS:0;jenkins-hbase4:33713] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:48,522 INFO [RS:0;jenkins-hbase4:33713] regionserver.LeaseManager(133): Closed leases 2023-02-11 15:00:48,522 INFO [RS:0;jenkins-hbase4:33713] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-11 15:00:48,522 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 15:00:48,523 INFO [RS:0;jenkins-hbase4:33713] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33713 2023-02-11 15:00:48,526 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33713,1676127645277 2023-02-11 15:00:48,526 ERROR [Listener at localhost/36191-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5f914ba4 rejected from java.util.concurrent.ThreadPoolExecutor@5baec48e[Terminated, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:48,709 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:48,709 INFO [RS:0;jenkins-hbase4:33713] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33713,1676127645277; zookeeper connection closed. 2023-02-11 15:00:48,709 ERROR [Listener at localhost/36191-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7ae134eb rejected from java.util.concurrent.ThreadPoolExecutor@5baec48e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:48,709 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:33713-0x10043e6106d0001, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:48,709 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6b903412] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6b903412 2023-02-11 15:00:48,709 ERROR [Listener at localhost/36191-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@25fb7bd rejected from java.util.concurrent.ThreadPoolExecutor@5baec48e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:48,809 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:48,809 INFO [M:0;jenkins-hbase4:37295] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37295,1676127644151; zookeeper connection closed. 2023-02-11 15:00:48,809 ERROR [Listener at localhost/36191-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@647265d6 rejected from java.util.concurrent.ThreadPoolExecutor@987471d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 24] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:48,809 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): master:37295-0x10043e6106d0000, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:48,809 ERROR [Listener at localhost/36191-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@56a45812 rejected from java.util.concurrent.ThreadPoolExecutor@987471d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 24] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:48,909 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:48,909 INFO [RS:1;jenkins-hbase4:37217] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37217,1676127645321; zookeeper connection closed. 2023-02-11 15:00:48,909 ERROR [Listener at localhost/36191-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@655cde86 rejected from java.util.concurrent.ThreadPoolExecutor@103d39bc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:48,910 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7c034770] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7c034770 2023-02-11 15:00:48,910 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:37217-0x10043e6106d0002, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:48,910 ERROR [Listener at localhost/36191-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1d2915b7 rejected from java.util.concurrent.ThreadPoolExecutor@103d39bc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:49,009 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:49,009 INFO [RS:2;jenkins-hbase4:46559] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,46559,1676127645351; zookeeper connection closed. 2023-02-11 15:00:49,010 ERROR [Listener at localhost/36191-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@19f35d47 rejected from java.util.concurrent.ThreadPoolExecutor@585e641a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:49,010 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7c5c14f5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7c5c14f5 2023-02-11 15:00:49,010 DEBUG [Listener at localhost/36191-EventThread] zookeeper.ZKWatcher(600): regionserver:46559-0x10043e6106d0003, quorum=127.0.0.1:63275, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:49,010 ERROR [Listener at localhost/36191-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1cdac88f rejected from java.util.concurrent.ThreadPoolExecutor@585e641a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:49,010 INFO [Listener at localhost/36191] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-11 15:00:49,011 WARN [Listener at localhost/36191] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 15:00:49,014 INFO [Listener at localhost/36191] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 15:00:49,117 WARN [BP-399959630-172.31.14.131-1676127641071 heartbeating to localhost/127.0.0.1:38847] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-11 15:00:49,117 WARN [BP-399959630-172.31.14.131-1676127641071 heartbeating to localhost/127.0.0.1:38847] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-399959630-172.31.14.131-1676127641071 (Datanode Uuid f1797d6a-658b-43e2-aba7-ea82a7c47cab) service to localhost/127.0.0.1:38847 2023-02-11 15:00:49,119 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/cluster_481e5d89-aaa2-47ab-8331-fe2aab94c5ba/dfs/data/data5/current/BP-399959630-172.31.14.131-1676127641071] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 15:00:49,119 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/cluster_481e5d89-aaa2-47ab-8331-fe2aab94c5ba/dfs/data/data6/current/BP-399959630-172.31.14.131-1676127641071] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 15:00:49,120 WARN [Listener at localhost/36191] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 15:00:49,122 INFO [Listener at localhost/36191] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 15:00:49,223 WARN [BP-399959630-172.31.14.131-1676127641071 heartbeating to localhost/127.0.0.1:38847] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-11 15:00:49,223 WARN [BP-399959630-172.31.14.131-1676127641071 heartbeating to localhost/127.0.0.1:38847] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-399959630-172.31.14.131-1676127641071 (Datanode Uuid eeb91041-2f31-4b32-9e1c-18e651680daa) service to localhost/127.0.0.1:38847 2023-02-11 15:00:49,224 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/cluster_481e5d89-aaa2-47ab-8331-fe2aab94c5ba/dfs/data/data3/current/BP-399959630-172.31.14.131-1676127641071] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 15:00:49,224 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/cluster_481e5d89-aaa2-47ab-8331-fe2aab94c5ba/dfs/data/data4/current/BP-399959630-172.31.14.131-1676127641071] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 15:00:49,225 WARN [Listener at localhost/36191] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 15:00:49,227 INFO [Listener at localhost/36191] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 15:00:49,328 WARN [BP-399959630-172.31.14.131-1676127641071 heartbeating to localhost/127.0.0.1:38847] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-11 15:00:49,329 WARN [BP-399959630-172.31.14.131-1676127641071 heartbeating to localhost/127.0.0.1:38847] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-399959630-172.31.14.131-1676127641071 (Datanode Uuid cfa573b9-160e-4f69-9a79-1d6bc607406f) service to localhost/127.0.0.1:38847 2023-02-11 15:00:49,329 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/cluster_481e5d89-aaa2-47ab-8331-fe2aab94c5ba/dfs/data/data1/current/BP-399959630-172.31.14.131-1676127641071] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 15:00:49,330 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/cluster_481e5d89-aaa2-47ab-8331-fe2aab94c5ba/dfs/data/data2/current/BP-399959630-172.31.14.131-1676127641071] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 15:00:49,354 INFO [Listener at localhost/36191] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 15:00:49,467 INFO [Listener at localhost/36191] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-11 15:00:49,506 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-02-11 15:00:49,518 INFO [Listener at localhost/36191] hbase.ResourceChecker(175): after: client.TestAsyncClusterAdminApi2#testStop Thread=77 (was 8) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-5-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Client (288607626) connection to localhost/127.0.0.1:38847 from jenkins.hfs.0 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: ForkJoinPool-2-worker-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:38847 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: Listener at localhost/36191 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:38847 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:38847 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (288607626) connection to localhost/127.0.0.1:38847 from jenkins.hfs.1 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:38847 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-5-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:300) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@3a67175f java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (288607626) connection to localhost/127.0.0.1:38847 from jenkins.hfs.2 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-5-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (288607626) connection to localhost/127.0.0.1:38847 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: IPC Client (288607626) connection to localhost/127.0.0.1:38847 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: HFileArchiver-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'NameNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=488 (was 260) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=290 (was 239) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3682 (was 4246) 2023-02-11 15:00:49,527 INFO [Listener at localhost/36191] hbase.ResourceChecker(147): before: client.TestAsyncClusterAdminApi2#testShutdown Thread=77, OpenFileDescriptor=488, MaxFileDescriptor=60000, SystemLoadAverage=290, ProcessCount=168, AvailableMemoryMB=3682 2023-02-11 15:00:49,528 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-11 15:00:49,528 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/hadoop.log.dir so I do NOT create it in target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645 2023-02-11 15:00:49,528 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cdb6e882-a640-e18a-edb1-0c9aae266459/hadoop.tmp.dir so I do NOT create it in target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645 2023-02-11 15:00:49,528 INFO [Listener at localhost/36191] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/cluster_7d9fd2fa-b316-1451-d397-153eeeb5960e, deleteOnExit=true 2023-02-11 15:00:49,528 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-11 15:00:49,528 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/test.cache.data in system properties and HBase conf 2023-02-11 15:00:49,529 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/hadoop.tmp.dir in system properties and HBase conf 2023-02-11 15:00:49,529 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/hadoop.log.dir in system properties and HBase conf 2023-02-11 15:00:49,529 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-11 15:00:49,529 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-11 15:00:49,529 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-11 15:00:49,529 DEBUG [Listener at localhost/36191] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-11 15:00:49,529 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-11 15:00:49,530 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-11 15:00:49,530 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-11 15:00:49,530 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-11 15:00:49,530 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-11 15:00:49,530 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-11 15:00:49,530 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-11 15:00:49,530 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-11 15:00:49,530 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-11 15:00:49,530 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/nfs.dump.dir in system properties and HBase conf 2023-02-11 15:00:49,531 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/java.io.tmpdir in system properties and HBase conf 2023-02-11 15:00:49,531 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-11 15:00:49,531 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-11 15:00:49,531 INFO [Listener at localhost/36191] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-11 15:00:49,535 WARN [Listener at localhost/36191] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-11 15:00:49,536 WARN [Listener at localhost/36191] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-11 15:00:49,582 WARN [Listener at localhost/36191] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 15:00:49,584 INFO [Listener at localhost/36191] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 15:00:49,591 INFO [Listener at localhost/36191] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/java.io.tmpdir/Jetty_localhost_42029_hdfs____.dm055p/webapp 2023-02-11 15:00:49,693 INFO [Listener at localhost/36191] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42029 2023-02-11 15:00:49,697 WARN [Listener at localhost/36191] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-11 15:00:49,697 WARN [Listener at localhost/36191] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-11 15:00:49,738 WARN [Listener at localhost/46043] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 15:00:49,749 WARN [Listener at localhost/46043] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 15:00:49,751 WARN [Listener at localhost/46043] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 15:00:49,752 INFO [Listener at localhost/46043] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 15:00:49,757 INFO [Listener at localhost/46043] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/java.io.tmpdir/Jetty_localhost_37243_datanode____4jry5w/webapp 2023-02-11 15:00:49,847 INFO [Listener at localhost/46043] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37243 2023-02-11 15:00:49,854 WARN [Listener at localhost/38085] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 15:00:49,872 WARN [Listener at localhost/38085] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 15:00:49,874 WARN [Listener at localhost/38085] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 15:00:49,875 INFO [Listener at localhost/38085] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 15:00:49,880 INFO [Listener at localhost/38085] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/java.io.tmpdir/Jetty_localhost_46187_datanode____smahw5/webapp 2023-02-11 15:00:49,970 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x860f09d6e5665e27: Processing first storage report for DS-b570b327-d659-49f7-95e3-9dc74572b52b from datanode 2cad4e84-73ab-4c81-8359-9945702008d5 2023-02-11 15:00:49,971 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x860f09d6e5665e27: from storage DS-b570b327-d659-49f7-95e3-9dc74572b52b node DatanodeRegistration(127.0.0.1:38297, datanodeUuid=2cad4e84-73ab-4c81-8359-9945702008d5, infoPort=34093, infoSecurePort=0, ipcPort=38085, storageInfo=lv=-57;cid=testClusterID;nsid=1879887812;c=1676127649538), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-11 15:00:49,971 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x860f09d6e5665e27: Processing first storage report for DS-99dbf689-6dab-4877-8512-610e9253067f from datanode 2cad4e84-73ab-4c81-8359-9945702008d5 2023-02-11 15:00:49,971 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x860f09d6e5665e27: from storage DS-99dbf689-6dab-4877-8512-610e9253067f node DatanodeRegistration(127.0.0.1:38297, datanodeUuid=2cad4e84-73ab-4c81-8359-9945702008d5, infoPort=34093, infoSecurePort=0, ipcPort=38085, storageInfo=lv=-57;cid=testClusterID;nsid=1879887812;c=1676127649538), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 15:00:49,994 INFO [Listener at localhost/38085] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46187 2023-02-11 15:00:50,005 WARN [Listener at localhost/39255] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 15:00:50,027 WARN [Listener at localhost/39255] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 15:00:50,030 WARN [Listener at localhost/39255] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 15:00:50,031 INFO [Listener at localhost/39255] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 15:00:50,035 INFO [Listener at localhost/39255] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/java.io.tmpdir/Jetty_localhost_46543_datanode____.vw3bdz/webapp 2023-02-11 15:00:50,119 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb42facd4c47b0c47: Processing first storage report for DS-cb02cf6c-530c-4fc8-90fa-f9bf61e62de1 from datanode 6d7ccb91-66ad-402c-ad88-d7e0b0744b5c 2023-02-11 15:00:50,119 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb42facd4c47b0c47: from storage DS-cb02cf6c-530c-4fc8-90fa-f9bf61e62de1 node DatanodeRegistration(127.0.0.1:45679, datanodeUuid=6d7ccb91-66ad-402c-ad88-d7e0b0744b5c, infoPort=33515, infoSecurePort=0, ipcPort=39255, storageInfo=lv=-57;cid=testClusterID;nsid=1879887812;c=1676127649538), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 15:00:50,119 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb42facd4c47b0c47: Processing first storage report for DS-0f681562-4073-4a80-82ad-7036aec259a1 from datanode 6d7ccb91-66ad-402c-ad88-d7e0b0744b5c 2023-02-11 15:00:50,119 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb42facd4c47b0c47: from storage DS-0f681562-4073-4a80-82ad-7036aec259a1 node DatanodeRegistration(127.0.0.1:45679, datanodeUuid=6d7ccb91-66ad-402c-ad88-d7e0b0744b5c, infoPort=33515, infoSecurePort=0, ipcPort=39255, storageInfo=lv=-57;cid=testClusterID;nsid=1879887812;c=1676127649538), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 15:00:50,141 INFO [Listener at localhost/39255] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46543 2023-02-11 15:00:50,149 WARN [Listener at localhost/38557] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 15:00:50,254 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x923a2fd530bc306a: Processing first storage report for DS-a30769f5-aa88-4e7b-87ae-5f13d9edd338 from datanode ee949f08-b0ab-4727-a105-5b15a196a5a3 2023-02-11 15:00:50,254 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x923a2fd530bc306a: from storage DS-a30769f5-aa88-4e7b-87ae-5f13d9edd338 node DatanodeRegistration(127.0.0.1:42269, datanodeUuid=ee949f08-b0ab-4727-a105-5b15a196a5a3, infoPort=40429, infoSecurePort=0, ipcPort=38557, storageInfo=lv=-57;cid=testClusterID;nsid=1879887812;c=1676127649538), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 15:00:50,254 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x923a2fd530bc306a: Processing first storage report for DS-2e4f7fe0-b433-4fb4-8b50-0c973f7690b0 from datanode ee949f08-b0ab-4727-a105-5b15a196a5a3 2023-02-11 15:00:50,254 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x923a2fd530bc306a: from storage DS-2e4f7fe0-b433-4fb4-8b50-0c973f7690b0 node DatanodeRegistration(127.0.0.1:42269, datanodeUuid=ee949f08-b0ab-4727-a105-5b15a196a5a3, infoPort=40429, infoSecurePort=0, ipcPort=38557, storageInfo=lv=-57;cid=testClusterID;nsid=1879887812;c=1676127649538), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 15:00:50,259 DEBUG [Listener at localhost/38557] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645 2023-02-11 15:00:50,262 INFO [Listener at localhost/38557] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/cluster_7d9fd2fa-b316-1451-d397-153eeeb5960e/zookeeper_0, clientPort=55993, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/cluster_7d9fd2fa-b316-1451-d397-153eeeb5960e/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/cluster_7d9fd2fa-b316-1451-d397-153eeeb5960e/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-11 15:00:50,264 INFO [Listener at localhost/38557] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=55993 2023-02-11 15:00:50,264 INFO [Listener at localhost/38557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:50,265 INFO [Listener at localhost/38557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:50,288 INFO [Listener at localhost/38557] util.FSUtils(479): Created version file at hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb with version=8 2023-02-11 15:00:50,289 INFO [Listener at localhost/38557] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:38847/user/jenkins/test-data/9990d903-8c73-a306-baa1-76e018962e46/hbase-staging 2023-02-11 15:00:50,290 INFO [Listener at localhost/38557] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=6 2023-02-11 15:00:50,291 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:50,291 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:50,291 INFO [Listener at localhost/38557] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 15:00:50,291 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:50,291 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 15:00:50,291 INFO [Listener at localhost/38557] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 15:00:50,293 INFO [Listener at localhost/38557] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37777 2023-02-11 15:00:50,293 INFO [Listener at localhost/38557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:50,295 INFO [Listener at localhost/38557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:50,296 INFO [Listener at localhost/38557] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37777 connecting to ZooKeeper ensemble=127.0.0.1:55993 2023-02-11 15:00:50,307 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:377770x0, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 15:00:50,309 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:37777-0x10043e62b9a0000 connected 2023-02-11 15:00:50,330 DEBUG [Listener at localhost/38557] zookeeper.ZKUtil(164): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 15:00:50,331 DEBUG [Listener at localhost/38557] zookeeper.ZKUtil(164): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 15:00:50,331 DEBUG [Listener at localhost/38557] zookeeper.ZKUtil(164): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 15:00:50,332 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37777 2023-02-11 15:00:50,336 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37777 2023-02-11 15:00:50,339 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37777 2023-02-11 15:00:50,339 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37777 2023-02-11 15:00:50,339 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37777 2023-02-11 15:00:50,340 INFO [Listener at localhost/38557] master.HMaster(439): hbase.rootdir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb, hbase.cluster.distributed=false 2023-02-11 15:00:50,353 INFO [Listener at localhost/38557] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-11 15:00:50,354 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:50,354 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:50,354 INFO [Listener at localhost/38557] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 15:00:50,354 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:50,354 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 15:00:50,354 INFO [Listener at localhost/38557] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 15:00:50,355 INFO [Listener at localhost/38557] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40967 2023-02-11 15:00:50,356 INFO [Listener at localhost/38557] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 15:00:50,357 DEBUG [Listener at localhost/38557] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 15:00:50,357 INFO [Listener at localhost/38557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:50,358 INFO [Listener at localhost/38557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:50,359 INFO [Listener at localhost/38557] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40967 connecting to ZooKeeper ensemble=127.0.0.1:55993 2023-02-11 15:00:50,363 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:409670x0, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 15:00:50,364 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:40967-0x10043e62b9a0001 connected 2023-02-11 15:00:50,364 DEBUG [Listener at localhost/38557] zookeeper.ZKUtil(164): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 15:00:50,364 DEBUG [Listener at localhost/38557] zookeeper.ZKUtil(164): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 15:00:50,365 DEBUG [Listener at localhost/38557] zookeeper.ZKUtil(164): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 15:00:50,368 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40967 2023-02-11 15:00:50,370 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40967 2023-02-11 15:00:50,370 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40967 2023-02-11 15:00:50,370 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40967 2023-02-11 15:00:50,371 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40967 2023-02-11 15:00:50,382 INFO [Listener at localhost/38557] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-11 15:00:50,383 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:50,383 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:50,383 INFO [Listener at localhost/38557] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 15:00:50,383 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:50,383 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 15:00:50,383 INFO [Listener at localhost/38557] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 15:00:50,385 INFO [Listener at localhost/38557] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41025 2023-02-11 15:00:50,385 INFO [Listener at localhost/38557] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 15:00:50,386 DEBUG [Listener at localhost/38557] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 15:00:50,387 INFO [Listener at localhost/38557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:50,388 INFO [Listener at localhost/38557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:50,389 INFO [Listener at localhost/38557] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41025 connecting to ZooKeeper ensemble=127.0.0.1:55993 2023-02-11 15:00:50,392 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:410250x0, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 15:00:50,393 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:41025-0x10043e62b9a0002 connected 2023-02-11 15:00:50,393 DEBUG [Listener at localhost/38557] zookeeper.ZKUtil(164): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 15:00:50,394 DEBUG [Listener at localhost/38557] zookeeper.ZKUtil(164): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 15:00:50,395 DEBUG [Listener at localhost/38557] zookeeper.ZKUtil(164): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 15:00:50,396 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41025 2023-02-11 15:00:50,398 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41025 2023-02-11 15:00:50,399 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41025 2023-02-11 15:00:50,399 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41025 2023-02-11 15:00:50,399 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41025 2023-02-11 15:00:50,410 INFO [Listener at localhost/38557] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-11 15:00:50,410 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:50,411 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:50,411 INFO [Listener at localhost/38557] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 15:00:50,411 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 15:00:50,411 INFO [Listener at localhost/38557] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 15:00:50,411 INFO [Listener at localhost/38557] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 15:00:50,412 INFO [Listener at localhost/38557] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34801 2023-02-11 15:00:50,413 INFO [Listener at localhost/38557] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 15:00:50,413 DEBUG [Listener at localhost/38557] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 15:00:50,414 INFO [Listener at localhost/38557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:50,415 INFO [Listener at localhost/38557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:50,416 INFO [Listener at localhost/38557] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34801 connecting to ZooKeeper ensemble=127.0.0.1:55993 2023-02-11 15:00:50,419 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:348010x0, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 15:00:50,420 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:34801-0x10043e62b9a0003 connected 2023-02-11 15:00:50,420 DEBUG [Listener at localhost/38557] zookeeper.ZKUtil(164): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 15:00:50,421 DEBUG [Listener at localhost/38557] zookeeper.ZKUtil(164): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 15:00:50,421 DEBUG [Listener at localhost/38557] zookeeper.ZKUtil(164): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 15:00:50,422 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34801 2023-02-11 15:00:50,422 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34801 2023-02-11 15:00:50,422 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34801 2023-02-11 15:00:50,422 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34801 2023-02-11 15:00:50,423 DEBUG [Listener at localhost/38557] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34801 2023-02-11 15:00:50,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,37777,1676127650290 2023-02-11 15:00:50,426 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-11 15:00:50,426 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,37777,1676127650290 2023-02-11 15:00:50,428 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 15:00:50,428 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 15:00:50,428 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:50,428 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 15:00:50,428 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 15:00:50,430 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-11 15:00:50,430 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-11 15:00:50,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,37777,1676127650290 from backup master directory 2023-02-11 15:00:50,432 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,37777,1676127650290 2023-02-11 15:00:50,432 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 15:00:50,432 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-11 15:00:50,432 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,37777,1676127650290 2023-02-11 15:00:50,454 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/hbase.id with ID: 2011c0b2-fb06-480b-a2c7-2f9de00873e3 2023-02-11 15:00:50,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 15:00:50,473 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:50,489 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x38437376 to 127.0.0.1:55993 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 15:00:50,493 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@34700241, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 15:00:50,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-11 15:00:50,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-11 15:00:50,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 15:00:50,497 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/MasterData/data/master/store-tmp 2023-02-11 15:00:50,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 15:00:50,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-11 15:00:50,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:50,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:50,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-11 15:00:50,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:50,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:50,511 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676127650510Disabling compacts and flushes for region at 1676127650510Disabling writes for close at 1676127650510Writing region close event to WAL at 1676127650510Closed at 1676127650510 2023-02-11 15:00:50,512 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/MasterData/WALs/jenkins-hbase4.apache.org,37777,1676127650290 2023-02-11 15:00:50,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37777%2C1676127650290, suffix=, logDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/MasterData/WALs/jenkins-hbase4.apache.org,37777,1676127650290, archiveDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/MasterData/oldWALs, maxLogs=10 2023-02-11 15:00:50,533 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38297,DS-b570b327-d659-49f7-95e3-9dc74572b52b,DISK] 2023-02-11 15:00:50,534 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45679,DS-cb02cf6c-530c-4fc8-90fa-f9bf61e62de1,DISK] 2023-02-11 15:00:50,534 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42269,DS-a30769f5-aa88-4e7b-87ae-5f13d9edd338,DISK] 2023-02-11 15:00:50,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/MasterData/WALs/jenkins-hbase4.apache.org,37777,1676127650290/jenkins-hbase4.apache.org%2C37777%2C1676127650290.1676127650516 2023-02-11 15:00:50,538 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38297,DS-b570b327-d659-49f7-95e3-9dc74572b52b,DISK], DatanodeInfoWithStorage[127.0.0.1:42269,DS-a30769f5-aa88-4e7b-87ae-5f13d9edd338,DISK], DatanodeInfoWithStorage[127.0.0.1:45679,DS-cb02cf6c-530c-4fc8-90fa-f9bf61e62de1,DISK]] 2023-02-11 15:00:50,538 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-11 15:00:50,538 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 15:00:50,538 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 15:00:50,538 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 15:00:50,540 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-11 15:00:50,542 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-11 15:00:50,543 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-11 15:00:50,543 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:50,544 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-11 15:00:50,545 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-11 15:00:50,549 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 15:00:50,551 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 15:00:50,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69343771, jitterRate=0.033302709460258484}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 15:00:50,552 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676127650538Initializing all the Stores at 1676127650539 (+1 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676127650539Cleaning up temporary data from old regions at 1676127650546 (+7 ms)Cleaning up detritus from prior splits at 1676127650546Region opened successfully at 1676127650552 (+6 ms) 2023-02-11 15:00:50,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-11 15:00:50,553 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-11 15:00:50,553 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-11 15:00:50,553 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 15:00:50,554 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 15:00:50,554 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-02-11 15:00:50,554 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-11 15:00:50,555 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-11 15:00:50,557 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-11 15:00:50,567 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-11 15:00:50,567 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-11 15:00:50,567 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-11 15:00:50,568 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-11 15:00:50,568 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-11 15:00:50,570 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:50,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-11 15:00:50,571 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-11 15:00:50,572 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-11 15:00:50,573 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 15:00:50,573 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 15:00:50,573 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:50,573 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 15:00:50,573 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 15:00:50,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,37777,1676127650290, sessionid=0x10043e62b9a0000, setting cluster-up flag (Was=false) 2023-02-11 15:00:50,578 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:50,583 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-11 15:00:50,584 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37777,1676127650290 2023-02-11 15:00:50,588 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:50,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-11 15:00:50,594 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37777,1676127650290 2023-02-11 15:00:50,595 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/.hbase-snapshot/.tmp 2023-02-11 15:00:50,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-11 15:00:50,598 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 15:00:50,598 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 15:00:50,598 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 15:00:50,598 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 15:00:50,598 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-11 15:00:50,598 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,598 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 15:00:50,598 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,603 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676127680603 2023-02-11 15:00:50,603 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-11 15:00:50,604 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-11 15:00:50,604 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-11 15:00:50,604 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-11 15:00:50,604 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-11 15:00:50,604 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-11 15:00:50,604 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,605 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-11 15:00:50,605 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-11 15:00:50,605 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-11 15:00:50,605 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-11 15:00:50,605 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-11 15:00:50,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-11 15:00:50,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-11 15:00:50,607 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676127650606,5,FailOnTimeoutGroup] 2023-02-11 15:00:50,607 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-11 15:00:50,607 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676127650607,5,FailOnTimeoutGroup] 2023-02-11 15:00:50,607 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,607 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-11 15:00:50,607 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,607 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,624 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-11 15:00:50,625 INFO [RS:0;jenkins-hbase4:40967] regionserver.HRegionServer(952): ClusterId : 2011c0b2-fb06-480b-a2c7-2f9de00873e3 2023-02-11 15:00:50,625 INFO [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(952): ClusterId : 2011c0b2-fb06-480b-a2c7-2f9de00873e3 2023-02-11 15:00:50,625 INFO [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(952): ClusterId : 2011c0b2-fb06-480b-a2c7-2f9de00873e3 2023-02-11 15:00:50,625 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-11 15:00:50,627 DEBUG [RS:0;jenkins-hbase4:40967] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 15:00:50,627 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb 2023-02-11 15:00:50,627 DEBUG [RS:2;jenkins-hbase4:34801] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 15:00:50,627 DEBUG [RS:1;jenkins-hbase4:41025] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 15:00:50,630 DEBUG [RS:0;jenkins-hbase4:40967] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 15:00:50,630 DEBUG [RS:0;jenkins-hbase4:40967] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 15:00:50,630 DEBUG [RS:1;jenkins-hbase4:41025] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 15:00:50,630 DEBUG [RS:2;jenkins-hbase4:34801] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 15:00:50,630 DEBUG [RS:1;jenkins-hbase4:41025] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 15:00:50,630 DEBUG [RS:2;jenkins-hbase4:34801] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 15:00:50,633 DEBUG [RS:0;jenkins-hbase4:40967] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 15:00:50,634 DEBUG [RS:1;jenkins-hbase4:41025] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 15:00:50,634 DEBUG [RS:2;jenkins-hbase4:34801] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 15:00:50,634 DEBUG [RS:0;jenkins-hbase4:40967] zookeeper.ReadOnlyZKClient(139): Connect 0x4a04beba to 127.0.0.1:55993 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 15:00:50,636 DEBUG [RS:2;jenkins-hbase4:34801] zookeeper.ReadOnlyZKClient(139): Connect 0x6d16cd6e to 127.0.0.1:55993 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 15:00:50,636 DEBUG [RS:1;jenkins-hbase4:41025] zookeeper.ReadOnlyZKClient(139): Connect 0x52aef3f3 to 127.0.0.1:55993 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 15:00:50,646 DEBUG [RS:2;jenkins-hbase4:34801] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@63d12ebc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 15:00:50,646 DEBUG [RS:0;jenkins-hbase4:40967] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4e3b2322, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 15:00:50,646 DEBUG [RS:2;jenkins-hbase4:34801] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@67eea61e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 15:00:50,646 DEBUG [RS:1;jenkins-hbase4:41025] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5c6917d6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 15:00:50,647 DEBUG [RS:0;jenkins-hbase4:40967] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@55e6fc6c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 15:00:50,647 DEBUG [RS:1;jenkins-hbase4:41025] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d541ded, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 15:00:50,652 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 15:00:50,656 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-11 15:00:50,657 DEBUG [RS:0;jenkins-hbase4:40967] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:40967 2023-02-11 15:00:50,657 DEBUG [RS:2;jenkins-hbase4:34801] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:34801 2023-02-11 15:00:50,657 INFO [RS:0;jenkins-hbase4:40967] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 15:00:50,657 INFO [RS:2;jenkins-hbase4:34801] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 15:00:50,657 INFO [RS:2;jenkins-hbase4:34801] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 15:00:50,657 INFO [RS:0;jenkins-hbase4:40967] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 15:00:50,658 DEBUG [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 15:00:50,658 DEBUG [RS:0;jenkins-hbase4:40967] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 15:00:50,658 DEBUG [RS:1;jenkins-hbase4:41025] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:41025 2023-02-11 15:00:50,658 INFO [RS:1;jenkins-hbase4:41025] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 15:00:50,658 INFO [RS:1;jenkins-hbase4:41025] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 15:00:50,658 DEBUG [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 15:00:50,658 INFO [RS:0;jenkins-hbase4:40967] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37777,1676127650290 with isa=jenkins-hbase4.apache.org/172.31.14.131:40967, startcode=1676127650353 2023-02-11 15:00:50,658 INFO [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37777,1676127650290 with isa=jenkins-hbase4.apache.org/172.31.14.131:34801, startcode=1676127650410 2023-02-11 15:00:50,659 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/info 2023-02-11 15:00:50,659 INFO [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37777,1676127650290 with isa=jenkins-hbase4.apache.org/172.31.14.131:41025, startcode=1676127650382 2023-02-11 15:00:50,659 DEBUG [RS:0;jenkins-hbase4:40967] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 15:00:50,659 DEBUG [RS:2;jenkins-hbase4:34801] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 15:00:50,659 DEBUG [RS:1;jenkins-hbase4:41025] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 15:00:50,660 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-11 15:00:50,661 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:50,661 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-11 15:00:50,663 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/rep_barrier 2023-02-11 15:00:50,663 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-11 15:00:50,665 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:50,666 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-11 15:00:50,667 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45539, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 15:00:50,668 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54857, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 15:00:50,667 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54237, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 15:00:50,669 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37777] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:50,669 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37777] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40967,1676127650353 2023-02-11 15:00:50,669 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37777] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:50,670 DEBUG [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb 2023-02-11 15:00:50,670 DEBUG [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:46043 2023-02-11 15:00:50,670 DEBUG [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb 2023-02-11 15:00:50,670 DEBUG [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 15:00:50,670 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/table 2023-02-11 15:00:50,670 DEBUG [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:46043 2023-02-11 15:00:50,671 DEBUG [RS:0;jenkins-hbase4:40967] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb 2023-02-11 15:00:50,671 DEBUG [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 15:00:50,671 DEBUG [RS:0;jenkins-hbase4:40967] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:46043 2023-02-11 15:00:50,671 DEBUG [RS:0;jenkins-hbase4:40967] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 15:00:50,672 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-11 15:00:50,672 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:50,673 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:50,676 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740 2023-02-11 15:00:50,676 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740 2023-02-11 15:00:50,679 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-11 15:00:50,680 DEBUG [RS:1;jenkins-hbase4:41025] zookeeper.ZKUtil(162): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:50,680 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41025,1676127650382] 2023-02-11 15:00:50,680 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34801,1676127650410] 2023-02-11 15:00:50,680 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40967,1676127650353] 2023-02-11 15:00:50,680 WARN [RS:1;jenkins-hbase4:41025] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 15:00:50,680 DEBUG [RS:0;jenkins-hbase4:40967] zookeeper.ZKUtil(162): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40967,1676127650353 2023-02-11 15:00:50,680 INFO [RS:1;jenkins-hbase4:41025] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 15:00:50,681 DEBUG [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(1947): logDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/WALs/jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:50,681 WARN [RS:0;jenkins-hbase4:40967] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 15:00:50,680 DEBUG [RS:2;jenkins-hbase4:34801] zookeeper.ZKUtil(162): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:50,681 INFO [RS:0;jenkins-hbase4:40967] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 15:00:50,681 WARN [RS:2;jenkins-hbase4:34801] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 15:00:50,681 DEBUG [RS:0;jenkins-hbase4:40967] regionserver.HRegionServer(1947): logDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/WALs/jenkins-hbase4.apache.org,40967,1676127650353 2023-02-11 15:00:50,681 INFO [RS:2;jenkins-hbase4:34801] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 15:00:50,681 DEBUG [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(1947): logDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/WALs/jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:50,681 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-11 15:00:50,692 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 15:00:50,694 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61180880, jitterRate=-0.0883338451385498}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-11 15:00:50,694 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676127650652Initializing all the Stores at 1676127650654 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676127650654Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676127650656 (+2 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676127650656Cleaning up temporary data from old regions at 1676127650678 (+22 ms)Cleaning up detritus from prior splits at 1676127650678Region opened successfully at 1676127650694 (+16 ms) 2023-02-11 15:00:50,694 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-11 15:00:50,695 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-11 15:00:50,695 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-11 15:00:50,695 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-11 15:00:50,695 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-11 15:00:50,695 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-11 15:00:50,695 DEBUG [RS:0;jenkins-hbase4:40967] zookeeper.ZKUtil(162): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:50,696 DEBUG [RS:1;jenkins-hbase4:41025] zookeeper.ZKUtil(162): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:50,696 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676127650694Disabling compacts and flushes for region at 1676127650694Disabling writes for close at 1676127650695 (+1 ms)Writing region close event to WAL at 1676127650695Closed at 1676127650695 2023-02-11 15:00:50,696 DEBUG [RS:1;jenkins-hbase4:41025] zookeeper.ZKUtil(162): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:50,697 DEBUG [RS:2;jenkins-hbase4:34801] zookeeper.ZKUtil(162): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:50,697 DEBUG [RS:1;jenkins-hbase4:41025] zookeeper.ZKUtil(162): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40967,1676127650353 2023-02-11 15:00:50,698 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-11 15:00:50,698 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-11 15:00:50,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-11 15:00:50,701 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-11 15:00:50,701 DEBUG [RS:0;jenkins-hbase4:40967] zookeeper.ZKUtil(162): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:50,702 DEBUG [RS:2;jenkins-hbase4:34801] zookeeper.ZKUtil(162): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:50,702 DEBUG [RS:0;jenkins-hbase4:40967] zookeeper.ZKUtil(162): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40967,1676127650353 2023-02-11 15:00:50,703 DEBUG [RS:1;jenkins-hbase4:41025] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 15:00:50,703 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-11 15:00:50,703 INFO [RS:1;jenkins-hbase4:41025] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 15:00:50,703 DEBUG [RS:0;jenkins-hbase4:40967] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 15:00:50,705 DEBUG [RS:2;jenkins-hbase4:34801] zookeeper.ZKUtil(162): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40967,1676127650353 2023-02-11 15:00:50,706 INFO [RS:0;jenkins-hbase4:40967] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 15:00:50,707 INFO [RS:1;jenkins-hbase4:41025] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 15:00:50,707 INFO [RS:1;jenkins-hbase4:41025] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 15:00:50,707 INFO [RS:1;jenkins-hbase4:41025] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,708 INFO [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 15:00:50,708 DEBUG [RS:2;jenkins-hbase4:34801] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 15:00:50,710 INFO [RS:2;jenkins-hbase4:34801] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 15:00:50,710 INFO [RS:0;jenkins-hbase4:40967] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 15:00:50,711 INFO [RS:0;jenkins-hbase4:40967] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 15:00:50,711 INFO [RS:0;jenkins-hbase4:40967] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,712 INFO [RS:1;jenkins-hbase4:41025] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,712 INFO [RS:0;jenkins-hbase4:40967] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 15:00:50,713 DEBUG [RS:1;jenkins-hbase4:41025] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,713 DEBUG [RS:1;jenkins-hbase4:41025] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,713 DEBUG [RS:1;jenkins-hbase4:41025] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,713 DEBUG [RS:1;jenkins-hbase4:41025] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,713 DEBUG [RS:1;jenkins-hbase4:41025] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,713 DEBUG [RS:1;jenkins-hbase4:41025] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 15:00:50,713 DEBUG [RS:1;jenkins-hbase4:41025] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,713 DEBUG [RS:1;jenkins-hbase4:41025] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,713 DEBUG [RS:1;jenkins-hbase4:41025] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,713 DEBUG [RS:1;jenkins-hbase4:41025] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,716 INFO [RS:2;jenkins-hbase4:34801] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 15:00:50,720 INFO [RS:0;jenkins-hbase4:40967] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,722 DEBUG [RS:0;jenkins-hbase4:40967] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,723 INFO [RS:2;jenkins-hbase4:34801] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 15:00:50,723 DEBUG [RS:0;jenkins-hbase4:40967] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,723 INFO [RS:2;jenkins-hbase4:34801] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,723 INFO [RS:1;jenkins-hbase4:41025] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,723 DEBUG [RS:0;jenkins-hbase4:40967] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,723 INFO [RS:1;jenkins-hbase4:41025] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,723 DEBUG [RS:0;jenkins-hbase4:40967] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,723 INFO [RS:1;jenkins-hbase4:41025] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,723 DEBUG [RS:0;jenkins-hbase4:40967] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,723 INFO [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 15:00:50,724 DEBUG [RS:0;jenkins-hbase4:40967] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 15:00:50,724 DEBUG [RS:0;jenkins-hbase4:40967] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,724 DEBUG [RS:0;jenkins-hbase4:40967] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,724 DEBUG [RS:0;jenkins-hbase4:40967] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,724 DEBUG [RS:0;jenkins-hbase4:40967] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,725 INFO [RS:0;jenkins-hbase4:40967] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,725 INFO [RS:0;jenkins-hbase4:40967] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,725 INFO [RS:0;jenkins-hbase4:40967] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,728 INFO [RS:2;jenkins-hbase4:34801] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,728 DEBUG [RS:2;jenkins-hbase4:34801] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,728 DEBUG [RS:2;jenkins-hbase4:34801] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,728 DEBUG [RS:2;jenkins-hbase4:34801] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,728 DEBUG [RS:2;jenkins-hbase4:34801] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,728 DEBUG [RS:2;jenkins-hbase4:34801] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,728 DEBUG [RS:2;jenkins-hbase4:34801] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 15:00:50,729 DEBUG [RS:2;jenkins-hbase4:34801] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,729 DEBUG [RS:2;jenkins-hbase4:34801] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,729 DEBUG [RS:2;jenkins-hbase4:34801] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,729 DEBUG [RS:2;jenkins-hbase4:34801] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 15:00:50,737 INFO [RS:1;jenkins-hbase4:41025] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 15:00:50,737 INFO [RS:1;jenkins-hbase4:41025] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41025,1676127650382-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,740 INFO [RS:2;jenkins-hbase4:34801] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,740 INFO [RS:0;jenkins-hbase4:40967] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 15:00:50,740 INFO [RS:2;jenkins-hbase4:34801] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,740 INFO [RS:0;jenkins-hbase4:40967] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40967,1676127650353-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,740 INFO [RS:2;jenkins-hbase4:34801] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,754 INFO [RS:2;jenkins-hbase4:34801] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 15:00:50,754 INFO [RS:2;jenkins-hbase4:34801] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34801,1676127650410-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:50,754 INFO [RS:1;jenkins-hbase4:41025] regionserver.Replication(203): jenkins-hbase4.apache.org,41025,1676127650382 started 2023-02-11 15:00:50,754 INFO [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,41025,1676127650382, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41025, sessionid=0x10043e62b9a0002 2023-02-11 15:00:50,754 DEBUG [RS:1;jenkins-hbase4:41025] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 15:00:50,754 DEBUG [RS:1;jenkins-hbase4:41025] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:50,754 DEBUG [RS:1;jenkins-hbase4:41025] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41025,1676127650382' 2023-02-11 15:00:50,755 DEBUG [RS:1;jenkins-hbase4:41025] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 15:00:50,755 DEBUG [RS:1;jenkins-hbase4:41025] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 15:00:50,756 DEBUG [RS:1;jenkins-hbase4:41025] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 15:00:50,756 DEBUG [RS:1;jenkins-hbase4:41025] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 15:00:50,756 DEBUG [RS:1;jenkins-hbase4:41025] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:50,756 DEBUG [RS:1;jenkins-hbase4:41025] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41025,1676127650382' 2023-02-11 15:00:50,756 DEBUG [RS:1;jenkins-hbase4:41025] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 15:00:50,756 DEBUG [RS:1;jenkins-hbase4:41025] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 15:00:50,757 DEBUG [RS:1;jenkins-hbase4:41025] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 15:00:50,757 INFO [RS:1;jenkins-hbase4:41025] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 15:00:50,757 INFO [RS:1;jenkins-hbase4:41025] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 15:00:50,760 INFO [RS:0;jenkins-hbase4:40967] regionserver.Replication(203): jenkins-hbase4.apache.org,40967,1676127650353 started 2023-02-11 15:00:50,760 INFO [RS:0;jenkins-hbase4:40967] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,40967,1676127650353, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40967, sessionid=0x10043e62b9a0001 2023-02-11 15:00:50,760 DEBUG [RS:0;jenkins-hbase4:40967] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 15:00:50,760 DEBUG [RS:0;jenkins-hbase4:40967] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40967,1676127650353 2023-02-11 15:00:50,760 DEBUG [RS:0;jenkins-hbase4:40967] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40967,1676127650353' 2023-02-11 15:00:50,760 DEBUG [RS:0;jenkins-hbase4:40967] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 15:00:50,761 DEBUG [RS:0;jenkins-hbase4:40967] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 15:00:50,761 DEBUG [RS:0;jenkins-hbase4:40967] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 15:00:50,761 DEBUG [RS:0;jenkins-hbase4:40967] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 15:00:50,761 DEBUG [RS:0;jenkins-hbase4:40967] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40967,1676127650353 2023-02-11 15:00:50,761 DEBUG [RS:0;jenkins-hbase4:40967] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40967,1676127650353' 2023-02-11 15:00:50,761 DEBUG [RS:0;jenkins-hbase4:40967] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 15:00:50,762 DEBUG [RS:0;jenkins-hbase4:40967] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 15:00:50,762 DEBUG [RS:0;jenkins-hbase4:40967] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 15:00:50,762 INFO [RS:0;jenkins-hbase4:40967] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 15:00:50,762 INFO [RS:0;jenkins-hbase4:40967] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 15:00:50,767 INFO [RS:2;jenkins-hbase4:34801] regionserver.Replication(203): jenkins-hbase4.apache.org,34801,1676127650410 started 2023-02-11 15:00:50,767 INFO [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,34801,1676127650410, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34801, sessionid=0x10043e62b9a0003 2023-02-11 15:00:50,767 DEBUG [RS:2;jenkins-hbase4:34801] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 15:00:50,767 DEBUG [RS:2;jenkins-hbase4:34801] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:50,767 DEBUG [RS:2;jenkins-hbase4:34801] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34801,1676127650410' 2023-02-11 15:00:50,767 DEBUG [RS:2;jenkins-hbase4:34801] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 15:00:50,767 DEBUG [RS:2;jenkins-hbase4:34801] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 15:00:50,768 DEBUG [RS:2;jenkins-hbase4:34801] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 15:00:50,768 DEBUG [RS:2;jenkins-hbase4:34801] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 15:00:50,768 DEBUG [RS:2;jenkins-hbase4:34801] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:50,768 DEBUG [RS:2;jenkins-hbase4:34801] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34801,1676127650410' 2023-02-11 15:00:50,768 DEBUG [RS:2;jenkins-hbase4:34801] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 15:00:50,768 DEBUG [RS:2;jenkins-hbase4:34801] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 15:00:50,769 DEBUG [RS:2;jenkins-hbase4:34801] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 15:00:50,769 INFO [RS:2;jenkins-hbase4:34801] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 15:00:50,769 INFO [RS:2;jenkins-hbase4:34801] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 15:00:50,853 DEBUG [jenkins-hbase4:37777] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-11 15:00:50,853 DEBUG [jenkins-hbase4:37777] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 15:00:50,856 DEBUG [jenkins-hbase4:37777] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 15:00:50,856 DEBUG [jenkins-hbase4:37777] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 15:00:50,856 DEBUG [jenkins-hbase4:37777] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 15:00:50,856 DEBUG [jenkins-hbase4:37777] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 15:00:50,857 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34801,1676127650410, state=OPENING 2023-02-11 15:00:50,858 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-11 15:00:50,859 INFO [RS:1;jenkins-hbase4:41025] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41025%2C1676127650382, suffix=, logDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/WALs/jenkins-hbase4.apache.org,41025,1676127650382, archiveDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/oldWALs, maxLogs=32 2023-02-11 15:00:50,860 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:50,860 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-11 15:00:50,860 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34801,1676127650410}] 2023-02-11 15:00:50,866 INFO [RS:0;jenkins-hbase4:40967] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40967%2C1676127650353, suffix=, logDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/WALs/jenkins-hbase4.apache.org,40967,1676127650353, archiveDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/oldWALs, maxLogs=32 2023-02-11 15:00:50,871 INFO [RS:2;jenkins-hbase4:34801] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34801%2C1676127650410, suffix=, logDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/WALs/jenkins-hbase4.apache.org,34801,1676127650410, archiveDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/oldWALs, maxLogs=32 2023-02-11 15:00:50,886 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42269,DS-a30769f5-aa88-4e7b-87ae-5f13d9edd338,DISK] 2023-02-11 15:00:50,887 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45679,DS-cb02cf6c-530c-4fc8-90fa-f9bf61e62de1,DISK] 2023-02-11 15:00:50,888 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38297,DS-b570b327-d659-49f7-95e3-9dc74572b52b,DISK] 2023-02-11 15:00:50,899 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42269,DS-a30769f5-aa88-4e7b-87ae-5f13d9edd338,DISK] 2023-02-11 15:00:50,899 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45679,DS-cb02cf6c-530c-4fc8-90fa-f9bf61e62de1,DISK] 2023-02-11 15:00:50,899 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38297,DS-b570b327-d659-49f7-95e3-9dc74572b52b,DISK] 2023-02-11 15:00:50,901 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38297,DS-b570b327-d659-49f7-95e3-9dc74572b52b,DISK] 2023-02-11 15:00:50,901 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45679,DS-cb02cf6c-530c-4fc8-90fa-f9bf61e62de1,DISK] 2023-02-11 15:00:50,901 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42269,DS-a30769f5-aa88-4e7b-87ae-5f13d9edd338,DISK] 2023-02-11 15:00:50,902 INFO [RS:1;jenkins-hbase4:41025] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/WALs/jenkins-hbase4.apache.org,41025,1676127650382/jenkins-hbase4.apache.org%2C41025%2C1676127650382.1676127650860 2023-02-11 15:00:50,908 DEBUG [RS:1;jenkins-hbase4:41025] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42269,DS-a30769f5-aa88-4e7b-87ae-5f13d9edd338,DISK], DatanodeInfoWithStorage[127.0.0.1:38297,DS-b570b327-d659-49f7-95e3-9dc74572b52b,DISK], DatanodeInfoWithStorage[127.0.0.1:45679,DS-cb02cf6c-530c-4fc8-90fa-f9bf61e62de1,DISK]] 2023-02-11 15:00:50,909 INFO [RS:2;jenkins-hbase4:34801] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/WALs/jenkins-hbase4.apache.org,34801,1676127650410/jenkins-hbase4.apache.org%2C34801%2C1676127650410.1676127650872 2023-02-11 15:00:50,909 DEBUG [RS:2;jenkins-hbase4:34801] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38297,DS-b570b327-d659-49f7-95e3-9dc74572b52b,DISK], DatanodeInfoWithStorage[127.0.0.1:45679,DS-cb02cf6c-530c-4fc8-90fa-f9bf61e62de1,DISK], DatanodeInfoWithStorage[127.0.0.1:42269,DS-a30769f5-aa88-4e7b-87ae-5f13d9edd338,DISK]] 2023-02-11 15:00:50,909 INFO [RS:0;jenkins-hbase4:40967] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/WALs/jenkins-hbase4.apache.org,40967,1676127650353/jenkins-hbase4.apache.org%2C40967%2C1676127650353.1676127650867 2023-02-11 15:00:50,912 DEBUG [RS:0;jenkins-hbase4:40967] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45679,DS-cb02cf6c-530c-4fc8-90fa-f9bf61e62de1,DISK], DatanodeInfoWithStorage[127.0.0.1:42269,DS-a30769f5-aa88-4e7b-87ae-5f13d9edd338,DISK], DatanodeInfoWithStorage[127.0.0.1:38297,DS-b570b327-d659-49f7-95e3-9dc74572b52b,DISK]] 2023-02-11 15:00:51,016 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:51,017 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 15:00:51,019 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47050, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 15:00:51,024 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-11 15:00:51,024 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 15:00:51,026 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34801%2C1676127650410.meta, suffix=.meta, logDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/WALs/jenkins-hbase4.apache.org,34801,1676127650410, archiveDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/oldWALs, maxLogs=32 2023-02-11 15:00:51,041 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45679,DS-cb02cf6c-530c-4fc8-90fa-f9bf61e62de1,DISK] 2023-02-11 15:00:51,042 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38297,DS-b570b327-d659-49f7-95e3-9dc74572b52b,DISK] 2023-02-11 15:00:51,042 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42269,DS-a30769f5-aa88-4e7b-87ae-5f13d9edd338,DISK] 2023-02-11 15:00:51,045 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/WALs/jenkins-hbase4.apache.org,34801,1676127650410/jenkins-hbase4.apache.org%2C34801%2C1676127650410.meta.1676127651027.meta 2023-02-11 15:00:51,045 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45679,DS-cb02cf6c-530c-4fc8-90fa-f9bf61e62de1,DISK], DatanodeInfoWithStorage[127.0.0.1:38297,DS-b570b327-d659-49f7-95e3-9dc74572b52b,DISK], DatanodeInfoWithStorage[127.0.0.1:42269,DS-a30769f5-aa88-4e7b-87ae-5f13d9edd338,DISK]] 2023-02-11 15:00:51,045 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-11 15:00:51,045 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-11 15:00:51,045 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-11 15:00:51,048 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-11 15:00:51,048 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-11 15:00:51,048 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 15:00:51,048 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-11 15:00:51,048 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-11 15:00:51,050 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-11 15:00:51,051 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/info 2023-02-11 15:00:51,051 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/info 2023-02-11 15:00:51,052 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-11 15:00:51,052 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:51,053 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-11 15:00:51,054 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/rep_barrier 2023-02-11 15:00:51,054 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/rep_barrier 2023-02-11 15:00:51,054 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-11 15:00:51,055 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:51,055 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-11 15:00:51,056 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/table 2023-02-11 15:00:51,056 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/table 2023-02-11 15:00:51,056 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-11 15:00:51,057 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:51,058 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740 2023-02-11 15:00:51,060 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740 2023-02-11 15:00:51,064 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-11 15:00:51,065 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-11 15:00:51,066 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65165878, jitterRate=-0.028952747583389282}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-11 15:00:51,066 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676127651048Writing region info on filesystem at 1676127651048Initializing all the Stores at 1676127651050 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676127651050Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676127651050Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676127651050Cleaning up temporary data from old regions at 1676127651061 (+11 ms)Cleaning up detritus from prior splits at 1676127651063 (+2 ms)Running coprocessor post-open hooks at 1676127651066 (+3 ms)Region opened successfully at 1676127651066 2023-02-11 15:00:51,068 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676127651016 2023-02-11 15:00:51,073 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-11 15:00:51,073 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-11 15:00:51,074 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34801,1676127650410, state=OPEN 2023-02-11 15:00:51,081 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-11 15:00:51,081 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-11 15:00:51,084 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-11 15:00:51,084 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34801,1676127650410 in 221 msec 2023-02-11 15:00:51,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-11 15:00:51,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 385 msec 2023-02-11 15:00:51,090 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 493 msec 2023-02-11 15:00:51,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676127650426, completionTime=-1 2023-02-11 15:00:51,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-11 15:00:51,090 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 15:00:51,093 DEBUG [hconnection-0x3d8c274d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 15:00:51,094 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47066, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 15:00:51,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 15:00:51,097 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676127711097 2023-02-11 15:00:51,097 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676127771097 2023-02-11 15:00:51,097 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-11 15:00:51,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37777,1676127650290-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:51,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37777,1676127650290-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:51,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37777,1676127650290-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:51,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37777, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:51,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-11 15:00:51,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-11 15:00:51,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-11 15:00:51,105 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-11 15:00:51,106 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-11 15:00:51,107 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 15:00:51,108 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 15:00:51,111 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/.tmp/data/hbase/namespace/8091fb2eb232327114f5315c68c869e1 2023-02-11 15:00:51,111 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/.tmp/data/hbase/namespace/8091fb2eb232327114f5315c68c869e1 empty. 2023-02-11 15:00:51,112 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/.tmp/data/hbase/namespace/8091fb2eb232327114f5315c68c869e1 2023-02-11 15:00:51,112 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-11 15:00:51,131 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-11 15:00:51,132 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 8091fb2eb232327114f5315c68c869e1, NAME => 'hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/.tmp 2023-02-11 15:00:51,148 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 15:00:51,148 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 8091fb2eb232327114f5315c68c869e1, disabling compactions & flushes 2023-02-11 15:00:51,148 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. 2023-02-11 15:00:51,148 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. 2023-02-11 15:00:51,148 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. after waiting 0 ms 2023-02-11 15:00:51,148 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. 2023-02-11 15:00:51,148 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. 2023-02-11 15:00:51,148 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 8091fb2eb232327114f5315c68c869e1: Waiting for close lock at 1676127651148Disabling compacts and flushes for region at 1676127651148Disabling writes for close at 1676127651148Writing region close event to WAL at 1676127651148Closed at 1676127651148 2023-02-11 15:00:51,151 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 15:00:51,153 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676127651153"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676127651153"}]},"ts":"1676127651153"} 2023-02-11 15:00:51,156 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 15:00:51,157 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 15:00:51,158 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676127651157"}]},"ts":"1676127651157"} 2023-02-11 15:00:51,160 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-11 15:00:51,164 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 15:00:51,165 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 15:00:51,165 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 15:00:51,165 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 15:00:51,165 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 15:00:51,165 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=8091fb2eb232327114f5315c68c869e1, ASSIGN}] 2023-02-11 15:00:51,167 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=8091fb2eb232327114f5315c68c869e1, ASSIGN 2023-02-11 15:00:51,168 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=8091fb2eb232327114f5315c68c869e1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41025,1676127650382; forceNewPlan=false, retain=false 2023-02-11 15:00:51,319 INFO [jenkins-hbase4:37777] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 15:00:51,320 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=8091fb2eb232327114f5315c68c869e1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:51,320 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676127651320"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676127651320"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676127651320"}]},"ts":"1676127651320"} 2023-02-11 15:00:51,322 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 8091fb2eb232327114f5315c68c869e1, server=jenkins-hbase4.apache.org,41025,1676127650382}] 2023-02-11 15:00:51,476 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:51,476 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 15:00:51,478 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57020, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 15:00:51,483 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. 2023-02-11 15:00:51,483 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8091fb2eb232327114f5315c68c869e1, NAME => 'hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1.', STARTKEY => '', ENDKEY => ''} 2023-02-11 15:00:51,484 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 8091fb2eb232327114f5315c68c869e1 2023-02-11 15:00:51,484 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 15:00:51,484 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8091fb2eb232327114f5315c68c869e1 2023-02-11 15:00:51,484 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8091fb2eb232327114f5315c68c869e1 2023-02-11 15:00:51,485 INFO [StoreOpener-8091fb2eb232327114f5315c68c869e1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 8091fb2eb232327114f5315c68c869e1 2023-02-11 15:00:51,487 DEBUG [StoreOpener-8091fb2eb232327114f5315c68c869e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/namespace/8091fb2eb232327114f5315c68c869e1/info 2023-02-11 15:00:51,487 DEBUG [StoreOpener-8091fb2eb232327114f5315c68c869e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/namespace/8091fb2eb232327114f5315c68c869e1/info 2023-02-11 15:00:51,489 INFO [StoreOpener-8091fb2eb232327114f5315c68c869e1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8091fb2eb232327114f5315c68c869e1 columnFamilyName info 2023-02-11 15:00:51,489 INFO [StoreOpener-8091fb2eb232327114f5315c68c869e1-1] regionserver.HStore(310): Store=8091fb2eb232327114f5315c68c869e1/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 15:00:51,490 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/namespace/8091fb2eb232327114f5315c68c869e1 2023-02-11 15:00:51,492 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/namespace/8091fb2eb232327114f5315c68c869e1 2023-02-11 15:00:51,494 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8091fb2eb232327114f5315c68c869e1 2023-02-11 15:00:51,497 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/namespace/8091fb2eb232327114f5315c68c869e1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 15:00:51,498 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8091fb2eb232327114f5315c68c869e1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62756601, jitterRate=-0.06485377252101898}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 15:00:51,498 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8091fb2eb232327114f5315c68c869e1: Running coprocessor pre-open hook at 1676127651484Writing region info on filesystem at 1676127651484Initializing all the Stores at 1676127651485 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676127651485Cleaning up temporary data from old regions at 1676127651492 (+7 ms)Cleaning up detritus from prior splits at 1676127651492Running coprocessor post-open hooks at 1676127651498 (+6 ms)Region opened successfully at 1676127651498 2023-02-11 15:00:51,499 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1., pid=6, masterSystemTime=1676127651476 2023-02-11 15:00:51,504 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. 2023-02-11 15:00:51,505 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. 2023-02-11 15:00:51,506 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=8091fb2eb232327114f5315c68c869e1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:51,507 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676127651506"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676127651506"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676127651506"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676127651506"}]},"ts":"1676127651506"} 2023-02-11 15:00:51,512 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-11 15:00:51,512 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 8091fb2eb232327114f5315c68c869e1, server=jenkins-hbase4.apache.org,41025,1676127650382 in 187 msec 2023-02-11 15:00:51,515 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-11 15:00:51,516 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=8091fb2eb232327114f5315c68c869e1, ASSIGN in 347 msec 2023-02-11 15:00:51,517 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 15:00:51,518 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676127651517"}]},"ts":"1676127651517"} 2023-02-11 15:00:51,520 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-11 15:00:51,522 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 15:00:51,524 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 418 msec 2023-02-11 15:00:51,607 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-11 15:00:51,608 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-11 15:00:51,608 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:51,611 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 15:00:51,613 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57030, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 15:00:51,616 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-11 15:00:51,627 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 15:00:51,632 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 15 msec 2023-02-11 15:00:51,638 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-11 15:00:51,646 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 15:00:51,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 12 msec 2023-02-11 15:00:51,663 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-11 15:00:51,665 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-11 15:00:51,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 1.233sec 2023-02-11 15:00:51,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-11 15:00:51,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-11 15:00:51,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-11 15:00:51,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37777,1676127650290-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-11 15:00:51,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37777,1676127650290-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-11 15:00:51,667 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-11 15:00:51,725 DEBUG [Listener at localhost/38557] zookeeper.ReadOnlyZKClient(139): Connect 0x0aed714b to 127.0.0.1:55993 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 15:00:51,730 DEBUG [Listener at localhost/38557] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@d5dbd57, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 15:00:51,732 DEBUG [hconnection-0x18025a26-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 15:00:51,734 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47078, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 15:00:51,736 INFO [Listener at localhost/38557] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,37777,1676127650290 2023-02-11 15:00:51,736 DEBUG [Listener at localhost/38557] zookeeper.ReadOnlyZKClient(139): Connect 0x29d30243 to 127.0.0.1:55993 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 15:00:51,743 DEBUG [ReadOnlyZKClient-127.0.0.1:55993@0x29d30243] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@335be0db, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 15:00:51,744 DEBUG [Listener at localhost/38557] client.ConnectionUtils(586): Start fetching master stub from registry 2023-02-11 15:00:51,745 DEBUG [ReadOnlyZKClient-127.0.0.1:55993@0x29d30243] client.AsyncConnectionImpl(289): The fetched master address is jenkins-hbase4.apache.org,37777,1676127650290 2023-02-11 15:00:51,745 DEBUG [ReadOnlyZKClient-127.0.0.1:55993@0x29d30243] client.ConnectionUtils(594): The fetched master stub is org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$Stub@37646268 2023-02-11 15:00:51,748 DEBUG [ReadOnlyZKClient-127.0.0.1:55993@0x29d30243] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 15:00:51,750 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49290, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 15:00:51,750 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37777] master.MasterRpcServices(1560): Client=jenkins//172.31.14.131 shutdown 2023-02-11 15:00:51,750 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37777] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,37777,1676127650290 2023-02-11 15:00:51,752 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 15:00:51,752 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 15:00:51,752 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 15:00:51,752 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37777] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 15:00:51,752 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 15:00:51,752 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:51,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 15:00:51,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 15:00:51,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 15:00:51,753 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37777] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x38437376 to 127.0.0.1:55993 2023-02-11 15:00:51,754 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37777] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:51,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 15:00:51,821 INFO [RS:0;jenkins-hbase4:40967] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,40967,1676127650353' ***** 2023-02-11 15:00:51,821 INFO [RS:0;jenkins-hbase4:40967] regionserver.HRegionServer(2310): STOPPED: Exiting; cluster shutdown set and not carrying any regions 2023-02-11 15:00:51,822 INFO [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(1065): Closing user regions 2023-02-11 15:00:51,822 INFO [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(1065): Closing user regions 2023-02-11 15:00:51,822 INFO [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(3304): Received CLOSE for 8091fb2eb232327114f5315c68c869e1 2023-02-11 15:00:51,823 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8091fb2eb232327114f5315c68c869e1, disabling compactions & flushes 2023-02-11 15:00:51,823 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. 2023-02-11 15:00:51,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. 2023-02-11 15:00:51,824 INFO [RS:0;jenkins-hbase4:40967] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 15:00:51,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. after waiting 0 ms 2023-02-11 15:00:51,824 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 15:00:51,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. 2023-02-11 15:00:51,824 INFO [RS:0;jenkins-hbase4:40967] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 15:00:51,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 8091fb2eb232327114f5315c68c869e1 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-11 15:00:51,825 INFO [RS:0;jenkins-hbase4:40967] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 15:00:51,825 INFO [RS:0;jenkins-hbase4:40967] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,40967,1676127650353 2023-02-11 15:00:51,825 DEBUG [RS:0;jenkins-hbase4:40967] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4a04beba to 127.0.0.1:55993 2023-02-11 15:00:51,825 DEBUG [RS:0;jenkins-hbase4:40967] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:51,825 INFO [RS:0;jenkins-hbase4:40967] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,40967,1676127650353; all regions closed. 2023-02-11 15:00:51,829 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 15:00:51,833 DEBUG [RS:0;jenkins-hbase4:40967] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/oldWALs 2023-02-11 15:00:51,833 INFO [RS:0;jenkins-hbase4:40967] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40967%2C1676127650353:(num 1676127650867) 2023-02-11 15:00:51,833 DEBUG [RS:0;jenkins-hbase4:40967] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:51,833 INFO [RS:0;jenkins-hbase4:40967] regionserver.LeaseManager(133): Closed leases 2023-02-11 15:00:51,834 INFO [RS:0;jenkins-hbase4:40967] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-11 15:00:51,834 INFO [RS:0;jenkins-hbase4:40967] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 15:00:51,834 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 15:00:51,834 INFO [RS:0;jenkins-hbase4:40967] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 15:00:51,834 INFO [RS:0;jenkins-hbase4:40967] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 15:00:51,834 INFO [RS:0;jenkins-hbase4:40967] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40967 2023-02-11 15:00:51,839 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40967,1676127650353 2023-02-11 15:00:51,839 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40967,1676127650353 2023-02-11 15:00:51,839 ERROR [Listener at localhost/38557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@40219d03 rejected from java.util.concurrent.ThreadPoolExecutor@441e01df[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:51,839 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:51,840 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:51,840 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:51,839 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40967,1676127650353 2023-02-11 15:00:51,840 ERROR [Listener at localhost/38557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7cd226a6 rejected from java.util.concurrent.ThreadPoolExecutor@441e01df[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:51,840 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:51,841 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40967,1676127650353] 2023-02-11 15:00:51,842 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40967,1676127650353; numProcessing=1 2023-02-11 15:00:51,842 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:51,842 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:51,843 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,40967,1676127650353 already deleted, retry=false 2023-02-11 15:00:51,843 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:51,843 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40967,1676127650353 expired; onlineServers=2 2023-02-11 15:00:51,844 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:51,844 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,40967,1676127650353 znode expired, triggering replicatorRemoved event 2023-02-11 15:00:51,845 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,40967,1676127650353 znode expired, triggering replicatorRemoved event 2023-02-11 15:00:51,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:51,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:51,849 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:51,849 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:51,850 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/namespace/8091fb2eb232327114f5315c68c869e1/.tmp/info/0666e7e01b3941998bd9497dd027403d 2023-02-11 15:00:51,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/namespace/8091fb2eb232327114f5315c68c869e1/.tmp/info/0666e7e01b3941998bd9497dd027403d as hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/namespace/8091fb2eb232327114f5315c68c869e1/info/0666e7e01b3941998bd9497dd027403d 2023-02-11 15:00:51,868 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/namespace/8091fb2eb232327114f5315c68c869e1/info/0666e7e01b3941998bd9497dd027403d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 15:00:51,870 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 8091fb2eb232327114f5315c68c869e1 in 45ms, sequenceid=6, compaction requested=false 2023-02-11 15:00:51,878 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/namespace/8091fb2eb232327114f5315c68c869e1/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-11 15:00:51,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. 2023-02-11 15:00:51,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8091fb2eb232327114f5315c68c869e1: Waiting for close lock at 1676127651823Running coprocessor pre-close hooks at 1676127651823Disabling compacts and flushes for region at 1676127651823Disabling writes for close at 1676127651824 (+1 ms)Obtaining lock to block concurrent updates at 1676127651825 (+1 ms)Preparing flush snapshotting stores in 8091fb2eb232327114f5315c68c869e1 at 1676127651825Finished memstore snapshotting hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676127651825Flushing stores of hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. at 1676127651826 (+1 ms)Flushing 8091fb2eb232327114f5315c68c869e1/info: creating writer at 1676127651826Flushing 8091fb2eb232327114f5315c68c869e1/info: appending metadata at 1676127651833 (+7 ms)Flushing 8091fb2eb232327114f5315c68c869e1/info: closing flushed file at 1676127651833Flushing 8091fb2eb232327114f5315c68c869e1/info: reopening flushed file at 1676127651862 (+29 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 8091fb2eb232327114f5315c68c869e1 in 45ms, sequenceid=6, compaction requested=false at 1676127651870 (+8 ms)Writing region close event to WAL at 1676127651872 (+2 ms)Running coprocessor post-close hooks at 1676127651879 (+7 ms)Closed at 1676127651879 2023-02-11 15:00:51,880 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37777] assignment.AssignmentManager(1094): RegionServer CLOSED 8091fb2eb232327114f5315c68c869e1 2023-02-11 15:00:51,881 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676127651104.8091fb2eb232327114f5315c68c869e1. 2023-02-11 15:00:51,925 DEBUG [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(1084): Waiting on 1588230740 2023-02-11 15:00:51,925 INFO [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,41025,1676127650382' ***** 2023-02-11 15:00:51,925 INFO [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(2310): STOPPED: Exiting; cluster shutdown set and not carrying any regions 2023-02-11 15:00:51,927 INFO [RS:1;jenkins-hbase4:41025] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 15:00:51,927 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 15:00:51,927 INFO [RS:1;jenkins-hbase4:41025] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 15:00:51,928 INFO [RS:1;jenkins-hbase4:41025] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 15:00:51,928 INFO [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:51,928 DEBUG [RS:1;jenkins-hbase4:41025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x52aef3f3 to 127.0.0.1:55993 2023-02-11 15:00:51,928 DEBUG [RS:1;jenkins-hbase4:41025] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:51,928 INFO [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,41025,1676127650382; all regions closed. 2023-02-11 15:00:51,934 DEBUG [RS:1;jenkins-hbase4:41025] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/oldWALs 2023-02-11 15:00:51,934 INFO [RS:1;jenkins-hbase4:41025] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41025%2C1676127650382:(num 1676127650860) 2023-02-11 15:00:51,934 DEBUG [RS:1;jenkins-hbase4:41025] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:51,935 INFO [RS:1;jenkins-hbase4:41025] regionserver.LeaseManager(133): Closed leases 2023-02-11 15:00:51,935 INFO [RS:1;jenkins-hbase4:41025] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-11 15:00:51,935 INFO [RS:1;jenkins-hbase4:41025] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 15:00:51,935 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 15:00:51,935 INFO [RS:1;jenkins-hbase4:41025] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 15:00:51,935 INFO [RS:1;jenkins-hbase4:41025] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 15:00:51,936 INFO [RS:1;jenkins-hbase4:41025] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41025 2023-02-11 15:00:51,940 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:51,940 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:51,940 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:51,940 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41025,1676127650382 2023-02-11 15:00:51,940 ERROR [Listener at localhost/38557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5da68c68 rejected from java.util.concurrent.ThreadPoolExecutor@59990304[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:51,941 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:51,941 ERROR [Listener at localhost/38557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@44616f7c rejected from java.util.concurrent.ThreadPoolExecutor@59990304[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:51,942 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41025,1676127650382] 2023-02-11 15:00:51,942 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:51,942 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41025,1676127650382; numProcessing=2 2023-02-11 15:00:51,942 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,41025,1676127650382 znode expired, triggering replicatorRemoved event 2023-02-11 15:00:51,943 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41025,1676127650382 already deleted, retry=false 2023-02-11 15:00:51,943 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41025,1676127650382 expired; onlineServers=1 2023-02-11 15:00:51,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:51,955 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:51,955 INFO [RS:0;jenkins-hbase4:40967] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,40967,1676127650353; zookeeper connection closed. 2023-02-11 15:00:51,955 ERROR [Listener at localhost/38557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4f39a86 rejected from java.util.concurrent.ThreadPoolExecutor@441e01df[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:51,956 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:40967-0x10043e62b9a0001, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:51,956 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@e0bbb7a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@e0bbb7a 2023-02-11 15:00:51,956 ERROR [Listener at localhost/38557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@cb995e1 rejected from java.util.concurrent.ThreadPoolExecutor@441e01df[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:52,025 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 15:00:52,027 INFO [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,34801,1676127650410' ***** 2023-02-11 15:00:52,027 INFO [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(2310): STOPPED: Stopped; only catalog regions remaining online 2023-02-11 15:00:52,027 INFO [RS:2;jenkins-hbase4:34801] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 15:00:52,027 INFO [RS:2;jenkins-hbase4:34801] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 15:00:52,027 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 15:00:52,028 INFO [RS:2;jenkins-hbase4:34801] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 15:00:52,028 INFO [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:52,028 DEBUG [RS:2;jenkins-hbase4:34801] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6d16cd6e to 127.0.0.1:55993 2023-02-11 15:00:52,028 DEBUG [RS:2;jenkins-hbase4:34801] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:52,028 INFO [RS:2;jenkins-hbase4:34801] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 15:00:52,028 INFO [RS:2;jenkins-hbase4:34801] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 15:00:52,029 INFO [RS:2;jenkins-hbase4:34801] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 15:00:52,029 INFO [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-11 15:00:52,029 INFO [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-11 15:00:52,029 DEBUG [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-11 15:00:52,029 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-11 15:00:52,029 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-11 15:00:52,029 DEBUG [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-11 15:00:52,030 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-11 15:00:52,030 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-11 15:00:52,030 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-11 15:00:52,030 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-02-11 15:00:52,045 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 15:00:52,045 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/.tmp/info/51d026ef0f4a4e75a606943cddb3d95f 2023-02-11 15:00:52,066 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/.tmp/table/d1f89ae2b48f4d0ebb63232cb38b9673 2023-02-11 15:00:52,074 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/.tmp/info/51d026ef0f4a4e75a606943cddb3d95f as hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/info/51d026ef0f4a4e75a606943cddb3d95f 2023-02-11 15:00:52,080 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/info/51d026ef0f4a4e75a606943cddb3d95f, entries=10, sequenceid=9, filesize=5.9 K 2023-02-11 15:00:52,082 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/.tmp/table/d1f89ae2b48f4d0ebb63232cb38b9673 as hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/table/d1f89ae2b48f4d0ebb63232cb38b9673 2023-02-11 15:00:52,091 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/table/d1f89ae2b48f4d0ebb63232cb38b9673, entries=2, sequenceid=9, filesize=4.7 K 2023-02-11 15:00:52,093 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 63ms, sequenceid=9, compaction requested=false 2023-02-11 15:00:52,099 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-02-11 15:00:52,100 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-11 15:00:52,100 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-11 15:00:52,100 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676127652029Running coprocessor pre-close hooks at 1676127652029Disabling compacts and flushes for region at 1676127652029Disabling writes for close at 1676127652030 (+1 ms)Obtaining lock to block concurrent updates at 1676127652030Preparing flush snapshotting stores in 1588230740 at 1676127652030Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=1290, getHeapSize=2912, getOffHeapSize=0, getCellsCount=12 at 1676127652030Flushing stores of hbase:meta,,1.1588230740 at 1676127652031 (+1 ms)Flushing 1588230740/info: creating writer at 1676127652031Flushing 1588230740/info: appending metadata at 1676127652035 (+4 ms)Flushing 1588230740/info: closing flushed file at 1676127652035Flushing 1588230740/table: creating writer at 1676127652054 (+19 ms)Flushing 1588230740/table: appending metadata at 1676127652057 (+3 ms)Flushing 1588230740/table: closing flushed file at 1676127652057Flushing 1588230740/info: reopening flushed file at 1676127652074 (+17 ms)Flushing 1588230740/table: reopening flushed file at 1676127652082 (+8 ms)Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 63ms, sequenceid=9, compaction requested=false at 1676127652093 (+11 ms)Writing region close event to WAL at 1676127652096 (+3 ms)Running coprocessor post-close hooks at 1676127652100 (+4 ms)Closed at 1676127652100 2023-02-11 15:00:52,100 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-11 15:00:52,230 INFO [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,34801,1676127650410; all regions closed. 2023-02-11 15:00:52,236 DEBUG [RS:2;jenkins-hbase4:34801] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/oldWALs 2023-02-11 15:00:52,236 INFO [RS:2;jenkins-hbase4:34801] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34801%2C1676127650410.meta:.meta(num 1676127651027) 2023-02-11 15:00:52,241 DEBUG [RS:2;jenkins-hbase4:34801] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/oldWALs 2023-02-11 15:00:52,241 INFO [RS:2;jenkins-hbase4:34801] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34801%2C1676127650410:(num 1676127650872) 2023-02-11 15:00:52,241 DEBUG [RS:2;jenkins-hbase4:34801] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:52,241 INFO [RS:2;jenkins-hbase4:34801] regionserver.LeaseManager(133): Closed leases 2023-02-11 15:00:52,242 INFO [RS:2;jenkins-hbase4:34801] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-11 15:00:52,242 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 15:00:52,242 INFO [RS:2;jenkins-hbase4:34801] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34801 2023-02-11 15:00:52,247 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34801,1676127650410 2023-02-11 15:00:52,247 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:52,247 ERROR [Listener at localhost/38557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@42b0e019 rejected from java.util.concurrent.ThreadPoolExecutor@468f7c97[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:52,249 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 15:00:52,249 ERROR [Listener at localhost/38557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@77b11d9b rejected from java.util.concurrent.ThreadPoolExecutor@468f7c97[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:52,250 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34801,1676127650410] 2023-02-11 15:00:52,250 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34801,1676127650410; numProcessing=3 2023-02-11 15:00:52,251 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34801,1676127650410 already deleted, retry=false 2023-02-11 15:00:52,251 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34801,1676127650410 expired; onlineServers=0 2023-02-11 15:00:52,251 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37777,1676127650290' ***** 2023-02-11 15:00:52,251 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-11 15:00:52,251 DEBUG [M:0;jenkins-hbase4:37777] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@30d1f3c3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 15:00:52,251 INFO [M:0;jenkins-hbase4:37777] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37777,1676127650290 2023-02-11 15:00:52,251 INFO [M:0;jenkins-hbase4:37777] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37777,1676127650290; all regions closed. 2023-02-11 15:00:52,251 DEBUG [M:0;jenkins-hbase4:37777] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:52,252 DEBUG [M:0;jenkins-hbase4:37777] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-11 15:00:52,252 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-11 15:00:52,252 DEBUG [M:0;jenkins-hbase4:37777] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-11 15:00:52,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676127650607] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676127650607,5,FailOnTimeoutGroup] 2023-02-11 15:00:52,252 INFO [M:0;jenkins-hbase4:37777] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-11 15:00:52,252 INFO [M:0;jenkins-hbase4:37777] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-11 15:00:52,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676127650606] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676127650606,5,FailOnTimeoutGroup] 2023-02-11 15:00:52,253 INFO [M:0;jenkins-hbase4:37777] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-11 15:00:52,253 DEBUG [M:0;jenkins-hbase4:37777] master.HMaster(1502): Stopping service threads 2023-02-11 15:00:52,253 INFO [M:0;jenkins-hbase4:37777] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-11 15:00:52,253 ERROR [M:0;jenkins-hbase4:37777] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] 2023-02-11 15:00:52,253 INFO [M:0;jenkins-hbase4:37777] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 15:00:52,254 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-11 15:00:52,254 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-11 15:00:52,254 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 15:00:52,254 DEBUG [M:0;jenkins-hbase4:37777] zookeeper.ZKUtil(398): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-11 15:00:52,254 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 15:00:52,254 WARN [M:0;jenkins-hbase4:37777] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-11 15:00:52,254 INFO [M:0;jenkins-hbase4:37777] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 15:00:52,255 INFO [M:0;jenkins-hbase4:37777] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-11 15:00:52,255 DEBUG [M:0;jenkins-hbase4:37777] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-11 15:00:52,255 INFO [M:0;jenkins-hbase4:37777] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:52,255 DEBUG [M:0;jenkins-hbase4:37777] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:52,255 DEBUG [M:0;jenkins-hbase4:37777] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-11 15:00:52,255 DEBUG [M:0;jenkins-hbase4:37777] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:52,255 INFO [M:0;jenkins-hbase4:37777] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-02-11 15:00:52,256 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:52,256 INFO [RS:1;jenkins-hbase4:41025] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,41025,1676127650382; zookeeper connection closed. 2023-02-11 15:00:52,256 ERROR [Listener at localhost/38557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3cddc9ca rejected from java.util.concurrent.ThreadPoolExecutor@59990304[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:52,256 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:41025-0x10043e62b9a0002, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:52,257 ERROR [Listener at localhost/38557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3361f379 rejected from java.util.concurrent.ThreadPoolExecutor@59990304[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:52,257 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4ed51e37] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4ed51e37 2023-02-11 15:00:52,258 INFO [Listener at localhost/38557] client.AsyncConnectionImpl(207): Connection has been closed by Listener at localhost/38557. 2023-02-11 15:00:52,258 DEBUG [Listener at localhost/38557] client.AsyncConnectionImpl(232): Call stack: at java.lang.Thread.getStackTrace(Thread.java:1564) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:209) at org.apache.hbase.thirdparty.com.google.common.io.Closeables.close(Closeables.java:79) at org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2.tearDown(TestAsyncClusterAdminApi2.java:75) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2023-02-11 15:00:52,258 DEBUG [Listener at localhost/38557] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:52,259 DEBUG [Listener at localhost/38557] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x29d30243 to 127.0.0.1:55993 2023-02-11 15:00:52,259 INFO [Listener at localhost/38557] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-11 15:00:52,259 DEBUG [Listener at localhost/38557] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0aed714b to 127.0.0.1:55993 2023-02-11 15:00:52,259 DEBUG [Listener at localhost/38557] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 15:00:52,260 DEBUG [Listener at localhost/38557] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-11 15:00:52,273 INFO [M:0;jenkins-hbase4:37777] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/244deb7021c24d74959711fb7935f9f8 2023-02-11 15:00:52,280 DEBUG [M:0;jenkins-hbase4:37777] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/244deb7021c24d74959711fb7935f9f8 as hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/244deb7021c24d74959711fb7935f9f8 2023-02-11 15:00:52,286 INFO [M:0;jenkins-hbase4:37777] regionserver.HStore(1080): Added hdfs://localhost:46043/user/jenkins/test-data/7db00c48-6a94-ae24-0bad-b8d86be071bb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/244deb7021c24d74959711fb7935f9f8, entries=8, sequenceid=66, filesize=6.3 K 2023-02-11 15:00:52,287 INFO [M:0;jenkins-hbase4:37777] regionserver.HRegion(2947): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 32ms, sequenceid=66, compaction requested=false 2023-02-11 15:00:52,288 INFO [M:0;jenkins-hbase4:37777] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 15:00:52,288 DEBUG [M:0;jenkins-hbase4:37777] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676127652255Disabling compacts and flushes for region at 1676127652255Disabling writes for close at 1676127652255Obtaining lock to block concurrent updates at 1676127652255Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676127652255Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=24646, getHeapSize=30248, getOffHeapSize=0, getCellsCount=71 at 1676127652256 (+1 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676127652257 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676127652258 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676127652263 (+5 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676127652263Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676127652281 (+18 ms)Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 32ms, sequenceid=66, compaction requested=false at 1676127652287 (+6 ms)Writing region close event to WAL at 1676127652288 (+1 ms)Closed at 1676127652288 2023-02-11 15:00:52,292 INFO [M:0;jenkins-hbase4:37777] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-11 15:00:52,292 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 15:00:52,292 INFO [M:0;jenkins-hbase4:37777] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37777 2023-02-11 15:00:52,295 DEBUG [M:0;jenkins-hbase4:37777] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,37777,1676127650290 already deleted, retry=false 2023-02-11 15:00:52,457 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:52,457 INFO [M:0;jenkins-hbase4:37777] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37777,1676127650290; zookeeper connection closed. 2023-02-11 15:00:52,457 ERROR [Listener at localhost/38557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@61352d7 rejected from java.util.concurrent.ThreadPoolExecutor@22cc6cd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 28] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:52,458 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): master:37777-0x10043e62b9a0000, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:52,458 ERROR [Listener at localhost/38557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6a3abcf1 rejected from java.util.concurrent.ThreadPoolExecutor@22cc6cd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 28] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:52,526 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-02-11 15:00:52,757 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:52,757 INFO [RS:2;jenkins-hbase4:34801] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,34801,1676127650410; zookeeper connection closed. 2023-02-11 15:00:52,757 ERROR [Listener at localhost/38557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@f922247 rejected from java.util.concurrent.ThreadPoolExecutor@468f7c97[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:52,758 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@c3fa97a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@c3fa97a 2023-02-11 15:00:52,758 DEBUG [Listener at localhost/38557-EventThread] zookeeper.ZKWatcher(600): regionserver:34801-0x10043e62b9a0003, quorum=127.0.0.1:55993, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 15:00:52,758 ERROR [Listener at localhost/38557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1c008147 rejected from java.util.concurrent.ThreadPoolExecutor@468f7c97[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 15:00:52,758 INFO [Listener at localhost/38557] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-11 15:00:52,759 WARN [Listener at localhost/38557] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 15:00:52,763 INFO [Listener at localhost/38557] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 15:00:52,866 WARN [BP-1723104026-172.31.14.131-1676127649538 heartbeating to localhost/127.0.0.1:46043] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-11 15:00:52,866 WARN [BP-1723104026-172.31.14.131-1676127649538 heartbeating to localhost/127.0.0.1:46043] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1723104026-172.31.14.131-1676127649538 (Datanode Uuid ee949f08-b0ab-4727-a105-5b15a196a5a3) service to localhost/127.0.0.1:46043 2023-02-11 15:00:52,867 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/cluster_7d9fd2fa-b316-1451-d397-153eeeb5960e/dfs/data/data5/current/BP-1723104026-172.31.14.131-1676127649538] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 15:00:52,867 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/cluster_7d9fd2fa-b316-1451-d397-153eeeb5960e/dfs/data/data6/current/BP-1723104026-172.31.14.131-1676127649538] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 15:00:52,868 WARN [Listener at localhost/38557] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 15:00:52,872 INFO [Listener at localhost/38557] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 15:00:52,976 WARN [BP-1723104026-172.31.14.131-1676127649538 heartbeating to localhost/127.0.0.1:46043] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-11 15:00:52,976 WARN [BP-1723104026-172.31.14.131-1676127649538 heartbeating to localhost/127.0.0.1:46043] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1723104026-172.31.14.131-1676127649538 (Datanode Uuid 6d7ccb91-66ad-402c-ad88-d7e0b0744b5c) service to localhost/127.0.0.1:46043 2023-02-11 15:00:52,977 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/cluster_7d9fd2fa-b316-1451-d397-153eeeb5960e/dfs/data/data3/current/BP-1723104026-172.31.14.131-1676127649538] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 15:00:52,977 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/cluster_7d9fd2fa-b316-1451-d397-153eeeb5960e/dfs/data/data4/current/BP-1723104026-172.31.14.131-1676127649538] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 15:00:52,978 WARN [Listener at localhost/38557] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 15:00:52,982 INFO [Listener at localhost/38557] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 15:00:53,085 WARN [BP-1723104026-172.31.14.131-1676127649538 heartbeating to localhost/127.0.0.1:46043] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-11 15:00:53,086 WARN [BP-1723104026-172.31.14.131-1676127649538 heartbeating to localhost/127.0.0.1:46043] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1723104026-172.31.14.131-1676127649538 (Datanode Uuid 2cad4e84-73ab-4c81-8359-9945702008d5) service to localhost/127.0.0.1:46043 2023-02-11 15:00:53,086 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/cluster_7d9fd2fa-b316-1451-d397-153eeeb5960e/dfs/data/data1/current/BP-1723104026-172.31.14.131-1676127649538] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 15:00:53,086 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/69a7ea4d-1b1d-9087-eab0-ea785abb5645/cluster_7d9fd2fa-b316-1451-d397-153eeeb5960e/dfs/data/data2/current/BP-1723104026-172.31.14.131-1676127649538] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 15:00:53,099 INFO [Listener at localhost/38557] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 15:00:53,214 INFO [Listener at localhost/38557] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-11 15:00:53,239 INFO [Listener at localhost/38557] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-02-11 15:00:53,250 INFO [Listener at localhost/38557] hbase.ResourceChecker(175): after: client.TestAsyncClusterAdminApi2#testShutdown Thread=107 (was 77) Potentially hanging thread: ReplicationExecutor-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager$NodeFailoverWorker.run(ReplicationSourceManager.java:703) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/38557 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:46043 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (288607626) connection to localhost/127.0.0.1:46043 from jenkins.hfs.4 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager$NodeFailoverWorker.run(ReplicationSourceManager.java:703) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (288607626) connection to localhost/127.0.0.1:46043 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:46043 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:46043 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (288607626) connection to localhost/127.0.0.1:46043 from jenkins.hfs.3 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: IPC Client (288607626) connection to localhost/127.0.0.1:46043 from jenkins.hfs.5 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:46043 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (288607626) connection to localhost/127.0.0.1:46043 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=529 (was 488) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=266 (was 290), ProcessCount=168 (was 168), AvailableMemoryMB=3591 (was 3682)