2023-02-09 10:00:13,116 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34 2023-02-09 10:00:13,129 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2 timeout: 13 mins 2023-02-09 10:00:13,164 INFO [Time-limited test] hbase.ResourceChecker(147): before: client.TestAsyncClusterAdminApi2#testStop Thread=8, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=301, ProcessCount=168, AvailableMemoryMB=3935 2023-02-09 10:00:13,170 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-09 10:00:13,171 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/cluster_1ad255c6-176f-12da-fe0a-3d2eb85088ac, deleteOnExit=true 2023-02-09 10:00:13,171 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-09 10:00:13,172 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/test.cache.data in system properties and HBase conf 2023-02-09 10:00:13,172 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/hadoop.tmp.dir in system properties and HBase conf 2023-02-09 10:00:13,172 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/hadoop.log.dir in system properties and HBase conf 2023-02-09 10:00:13,173 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-09 10:00:13,173 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-09 10:00:13,173 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-09 10:00:13,291 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-09 10:00:13,667 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-09 10:00:13,671 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-09 10:00:13,671 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-09 10:00:13,671 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-09 10:00:13,672 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 10:00:13,672 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-09 10:00:13,672 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-09 10:00:13,673 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 10:00:13,673 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 10:00:13,673 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-09 10:00:13,673 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/nfs.dump.dir in system properties and HBase conf 2023-02-09 10:00:13,674 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/java.io.tmpdir in system properties and HBase conf 2023-02-09 10:00:13,674 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 10:00:13,674 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-09 10:00:13,674 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-09 10:00:14,149 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 10:00:14,154 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 10:00:14,441 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-09 10:00:14,596 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-09 10:00:14,610 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 10:00:14,645 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 10:00:14,677 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/java.io.tmpdir/Jetty_localhost_37719_hdfs____a89rad/webapp 2023-02-09 10:00:14,853 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37719 2023-02-09 10:00:14,864 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 10:00:14,864 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 10:00:15,260 WARN [Listener at localhost/34347] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 10:00:15,354 WARN [Listener at localhost/34347] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 10:00:15,372 WARN [Listener at localhost/34347] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 10:00:15,380 INFO [Listener at localhost/34347] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 10:00:15,387 INFO [Listener at localhost/34347] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/java.io.tmpdir/Jetty_localhost_35983_datanode____kgduex/webapp 2023-02-09 10:00:15,486 INFO [Listener at localhost/34347] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35983 2023-02-09 10:00:15,798 WARN [Listener at localhost/40583] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 10:00:15,809 WARN [Listener at localhost/40583] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 10:00:15,813 WARN [Listener at localhost/40583] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 10:00:15,816 INFO [Listener at localhost/40583] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 10:00:15,823 INFO [Listener at localhost/40583] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/java.io.tmpdir/Jetty_localhost_44905_datanode____.xljwkb/webapp 2023-02-09 10:00:15,926 INFO [Listener at localhost/40583] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44905 2023-02-09 10:00:15,935 WARN [Listener at localhost/45627] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 10:00:15,947 WARN [Listener at localhost/45627] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 10:00:15,949 WARN [Listener at localhost/45627] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 10:00:15,951 INFO [Listener at localhost/45627] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 10:00:15,956 INFO [Listener at localhost/45627] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/java.io.tmpdir/Jetty_localhost_38541_datanode____9kqv6c/webapp 2023-02-09 10:00:16,072 INFO [Listener at localhost/45627] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38541 2023-02-09 10:00:16,081 WARN [Listener at localhost/33549] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 10:00:16,268 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x24ab1f47c399b05d: Processing first storage report for DS-8f7eed21-4181-4bee-9bca-b4550cc10328 from datanode 3e3369a7-c746-44e4-bf58-1e5d694f2bef 2023-02-09 10:00:16,270 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x24ab1f47c399b05d: from storage DS-8f7eed21-4181-4bee-9bca-b4550cc10328 node DatanodeRegistration(127.0.0.1:43439, datanodeUuid=3e3369a7-c746-44e4-bf58-1e5d694f2bef, infoPort=46603, infoSecurePort=0, ipcPort=40583, storageInfo=lv=-57;cid=testClusterID;nsid=1929054244;c=1675936814228), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-09 10:00:16,270 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6a4913cd3432a20b: Processing first storage report for DS-c5f29a4c-f95b-430e-9505-66d38f6639a3 from datanode 26b6dc11-b58f-4fc6-9b14-326fd99c9d1a 2023-02-09 10:00:16,270 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6a4913cd3432a20b: from storage DS-c5f29a4c-f95b-430e-9505-66d38f6639a3 node DatanodeRegistration(127.0.0.1:38331, datanodeUuid=26b6dc11-b58f-4fc6-9b14-326fd99c9d1a, infoPort=39853, infoSecurePort=0, ipcPort=33549, storageInfo=lv=-57;cid=testClusterID;nsid=1929054244;c=1675936814228), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 10:00:16,270 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6817150a652fe0a0: Processing first storage report for DS-74c4397f-ec33-44d3-8a28-a1b555a8ddbd from datanode a116a5ba-577f-46c2-a944-acb8054cda7f 2023-02-09 10:00:16,270 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6817150a652fe0a0: from storage DS-74c4397f-ec33-44d3-8a28-a1b555a8ddbd node DatanodeRegistration(127.0.0.1:37805, datanodeUuid=a116a5ba-577f-46c2-a944-acb8054cda7f, infoPort=44219, infoSecurePort=0, ipcPort=45627, storageInfo=lv=-57;cid=testClusterID;nsid=1929054244;c=1675936814228), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 10:00:16,271 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x24ab1f47c399b05d: Processing first storage report for DS-1757add7-514d-4406-9c2b-1d995090d529 from datanode 3e3369a7-c746-44e4-bf58-1e5d694f2bef 2023-02-09 10:00:16,271 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x24ab1f47c399b05d: from storage DS-1757add7-514d-4406-9c2b-1d995090d529 node DatanodeRegistration(127.0.0.1:43439, datanodeUuid=3e3369a7-c746-44e4-bf58-1e5d694f2bef, infoPort=46603, infoSecurePort=0, ipcPort=40583, storageInfo=lv=-57;cid=testClusterID;nsid=1929054244;c=1675936814228), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 10:00:16,271 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6a4913cd3432a20b: Processing first storage report for DS-c51f31d1-5ebe-4585-9d0a-c336b5ba74bd from datanode 26b6dc11-b58f-4fc6-9b14-326fd99c9d1a 2023-02-09 10:00:16,271 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6a4913cd3432a20b: from storage DS-c51f31d1-5ebe-4585-9d0a-c336b5ba74bd node DatanodeRegistration(127.0.0.1:38331, datanodeUuid=26b6dc11-b58f-4fc6-9b14-326fd99c9d1a, infoPort=39853, infoSecurePort=0, ipcPort=33549, storageInfo=lv=-57;cid=testClusterID;nsid=1929054244;c=1675936814228), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 10:00:16,271 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6817150a652fe0a0: Processing first storage report for DS-9a5e0bd7-6d44-4411-8d45-9dfc63285bdf from datanode a116a5ba-577f-46c2-a944-acb8054cda7f 2023-02-09 10:00:16,271 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6817150a652fe0a0: from storage DS-9a5e0bd7-6d44-4411-8d45-9dfc63285bdf node DatanodeRegistration(127.0.0.1:37805, datanodeUuid=a116a5ba-577f-46c2-a944-acb8054cda7f, infoPort=44219, infoSecurePort=0, ipcPort=45627, storageInfo=lv=-57;cid=testClusterID;nsid=1929054244;c=1675936814228), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 10:00:16,476 DEBUG [Listener at localhost/33549] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34 2023-02-09 10:00:16,538 INFO [Listener at localhost/33549] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/cluster_1ad255c6-176f-12da-fe0a-3d2eb85088ac/zookeeper_0, clientPort=51004, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/cluster_1ad255c6-176f-12da-fe0a-3d2eb85088ac/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/cluster_1ad255c6-176f-12da-fe0a-3d2eb85088ac/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-09 10:00:16,552 INFO [Listener at localhost/33549] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=51004 2023-02-09 10:00:16,563 INFO [Listener at localhost/33549] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:16,565 INFO [Listener at localhost/33549] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:17,235 INFO [Listener at localhost/33549] util.FSUtils(479): Created version file at hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6 with version=8 2023-02-09 10:00:17,235 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/hbase-staging 2023-02-09 10:00:17,548 INFO [Listener at localhost/33549] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-09 10:00:17,997 INFO [Listener at localhost/33549] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=6 2023-02-09 10:00:18,030 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:18,030 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:18,030 INFO [Listener at localhost/33549] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 10:00:18,031 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:18,031 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 10:00:18,169 INFO [Listener at localhost/33549] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 10:00:18,240 DEBUG [Listener at localhost/33549] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-09 10:00:18,336 INFO [Listener at localhost/33549] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42981 2023-02-09 10:00:18,346 INFO [Listener at localhost/33549] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:18,348 INFO [Listener at localhost/33549] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:18,368 INFO [Listener at localhost/33549] zookeeper.RecoverableZooKeeper(93): Process identifier=master:42981 connecting to ZooKeeper ensemble=127.0.0.1:51004 2023-02-09 10:00:18,415 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:429810x0, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 10:00:18,417 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:42981-0x100388647510000 connected 2023-02-09 10:00:18,441 DEBUG [Listener at localhost/33549] zookeeper.ZKUtil(164): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 10:00:18,442 DEBUG [Listener at localhost/33549] zookeeper.ZKUtil(164): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 10:00:18,446 DEBUG [Listener at localhost/33549] zookeeper.ZKUtil(164): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 10:00:18,454 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42981 2023-02-09 10:00:18,454 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42981 2023-02-09 10:00:18,455 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42981 2023-02-09 10:00:18,455 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42981 2023-02-09 10:00:18,456 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42981 2023-02-09 10:00:18,462 INFO [Listener at localhost/33549] master.HMaster(439): hbase.rootdir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6, hbase.cluster.distributed=false 2023-02-09 10:00:18,530 INFO [Listener at localhost/33549] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-09 10:00:18,531 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:18,531 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:18,531 INFO [Listener at localhost/33549] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 10:00:18,531 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:18,531 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 10:00:18,536 INFO [Listener at localhost/33549] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 10:00:18,539 INFO [Listener at localhost/33549] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33883 2023-02-09 10:00:18,541 INFO [Listener at localhost/33549] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 10:00:18,548 DEBUG [Listener at localhost/33549] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 10:00:18,549 INFO [Listener at localhost/33549] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:18,551 INFO [Listener at localhost/33549] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:18,553 INFO [Listener at localhost/33549] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33883 connecting to ZooKeeper ensemble=127.0.0.1:51004 2023-02-09 10:00:18,556 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:338830x0, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 10:00:18,557 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33883-0x100388647510001 connected 2023-02-09 10:00:18,558 DEBUG [Listener at localhost/33549] zookeeper.ZKUtil(164): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 10:00:18,559 DEBUG [Listener at localhost/33549] zookeeper.ZKUtil(164): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 10:00:18,559 DEBUG [Listener at localhost/33549] zookeeper.ZKUtil(164): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 10:00:18,560 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33883 2023-02-09 10:00:18,560 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33883 2023-02-09 10:00:18,561 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33883 2023-02-09 10:00:18,561 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33883 2023-02-09 10:00:18,561 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33883 2023-02-09 10:00:18,574 INFO [Listener at localhost/33549] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-09 10:00:18,574 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:18,574 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:18,574 INFO [Listener at localhost/33549] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 10:00:18,575 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:18,575 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 10:00:18,575 INFO [Listener at localhost/33549] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 10:00:18,576 INFO [Listener at localhost/33549] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39893 2023-02-09 10:00:18,577 INFO [Listener at localhost/33549] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 10:00:18,579 DEBUG [Listener at localhost/33549] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 10:00:18,580 INFO [Listener at localhost/33549] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:18,582 INFO [Listener at localhost/33549] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:18,584 INFO [Listener at localhost/33549] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39893 connecting to ZooKeeper ensemble=127.0.0.1:51004 2023-02-09 10:00:18,589 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:398930x0, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 10:00:18,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:39893-0x100388647510002 connected 2023-02-09 10:00:18,590 DEBUG [Listener at localhost/33549] zookeeper.ZKUtil(164): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 10:00:18,591 DEBUG [Listener at localhost/33549] zookeeper.ZKUtil(164): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 10:00:18,591 DEBUG [Listener at localhost/33549] zookeeper.ZKUtil(164): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 10:00:18,593 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39893 2023-02-09 10:00:18,593 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39893 2023-02-09 10:00:18,593 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39893 2023-02-09 10:00:18,594 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39893 2023-02-09 10:00:18,594 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39893 2023-02-09 10:00:18,607 INFO [Listener at localhost/33549] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-09 10:00:18,607 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:18,607 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:18,607 INFO [Listener at localhost/33549] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 10:00:18,607 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:18,607 INFO [Listener at localhost/33549] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 10:00:18,608 INFO [Listener at localhost/33549] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 10:00:18,609 INFO [Listener at localhost/33549] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36117 2023-02-09 10:00:18,609 INFO [Listener at localhost/33549] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 10:00:18,610 DEBUG [Listener at localhost/33549] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 10:00:18,611 INFO [Listener at localhost/33549] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:18,613 INFO [Listener at localhost/33549] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:18,614 INFO [Listener at localhost/33549] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36117 connecting to ZooKeeper ensemble=127.0.0.1:51004 2023-02-09 10:00:18,617 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:361170x0, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 10:00:18,618 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:36117-0x100388647510003 connected 2023-02-09 10:00:18,618 DEBUG [Listener at localhost/33549] zookeeper.ZKUtil(164): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 10:00:18,618 DEBUG [Listener at localhost/33549] zookeeper.ZKUtil(164): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 10:00:18,619 DEBUG [Listener at localhost/33549] zookeeper.ZKUtil(164): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 10:00:18,620 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36117 2023-02-09 10:00:18,620 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36117 2023-02-09 10:00:18,621 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36117 2023-02-09 10:00:18,621 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36117 2023-02-09 10:00:18,621 DEBUG [Listener at localhost/33549] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36117 2023-02-09 10:00:18,624 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,42981,1675936817383 2023-02-09 10:00:18,633 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 10:00:18,634 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,42981,1675936817383 2023-02-09 10:00:18,652 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 10:00:18,652 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 10:00:18,652 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 10:00:18,652 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 10:00:18,653 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:18,654 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 10:00:18,655 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,42981,1675936817383 from backup master directory 2023-02-09 10:00:18,656 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 10:00:18,660 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,42981,1675936817383 2023-02-09 10:00:18,660 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 10:00:18,661 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 10:00:18,661 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,42981,1675936817383 2023-02-09 10:00:18,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-09 10:00:18,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-09 10:00:18,755 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/hbase.id with ID: e9e9143f-54f9-4717-9ab8-f25d43186326 2023-02-09 10:00:18,797 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:18,812 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:18,859 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x041fbeaa to 127.0.0.1:51004 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 10:00:18,891 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6abc8392, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 10:00:18,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-09 10:00:18,915 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-09 10:00:18,932 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-09 10:00:18,932 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-09 10:00:18,935 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 10:00:18,940 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 10:00:18,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 10:00:18,975 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/MasterData/data/master/store-tmp 2023-02-09 10:00:19,035 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 10:00:19,036 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 10:00:19,036 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:19,036 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:19,036 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 10:00:19,036 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:19,036 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:19,037 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675936819035Disabling compacts and flushes for region at 1675936819036 (+1 ms)Disabling writes for close at 1675936819036Writing region close event to WAL at 1675936819036Closed at 1675936819036 2023-02-09 10:00:19,038 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/MasterData/WALs/jenkins-hbase4.apache.org,42981,1675936817383 2023-02-09 10:00:19,057 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42981%2C1675936817383, suffix=, logDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/MasterData/WALs/jenkins-hbase4.apache.org,42981,1675936817383, archiveDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/MasterData/oldWALs, maxLogs=10 2023-02-09 10:00:19,111 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38331,DS-c5f29a4c-f95b-430e-9505-66d38f6639a3,DISK] 2023-02-09 10:00:19,111 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37805,DS-74c4397f-ec33-44d3-8a28-a1b555a8ddbd,DISK] 2023-02-09 10:00:19,111 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43439,DS-8f7eed21-4181-4bee-9bca-b4550cc10328,DISK] 2023-02-09 10:00:19,119 DEBUG [RS-EventLoopGroup-5-1] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-09 10:00:19,185 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/MasterData/WALs/jenkins-hbase4.apache.org,42981,1675936817383/jenkins-hbase4.apache.org%2C42981%2C1675936817383.1675936819067 2023-02-09 10:00:19,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43439,DS-8f7eed21-4181-4bee-9bca-b4550cc10328,DISK], DatanodeInfoWithStorage[127.0.0.1:38331,DS-c5f29a4c-f95b-430e-9505-66d38f6639a3,DISK], DatanodeInfoWithStorage[127.0.0.1:37805,DS-74c4397f-ec33-44d3-8a28-a1b555a8ddbd,DISK]] 2023-02-09 10:00:19,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-09 10:00:19,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 10:00:19,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 10:00:19,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 10:00:19,246 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-09 10:00:19,253 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-09 10:00:19,277 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-09 10:00:19,289 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:19,295 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 10:00:19,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 10:00:19,311 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 10:00:19,315 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 10:00:19,316 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74681466, jitterRate=0.11284056305885315}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 10:00:19,316 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675936819216Initializing all the Stores at 1675936819218 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675936819219 (+1 ms)Cleaning up temporary data from old regions at 1675936819303 (+84 ms)Cleaning up detritus from prior splits at 1675936819303Region opened successfully at 1675936819316 (+13 ms) 2023-02-09 10:00:19,318 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-09 10:00:19,337 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-09 10:00:19,337 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-09 10:00:19,339 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 10:00:19,341 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-09 10:00:19,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-02-09 10:00:19,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-09 10:00:19,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-09 10:00:19,404 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-09 10:00:19,428 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-09 10:00:19,431 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-09 10:00:19,432 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-09 10:00:19,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-09 10:00:19,441 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-09 10:00:19,444 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:19,445 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-09 10:00:19,446 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-09 10:00:19,458 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-09 10:00:19,463 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 10:00:19,463 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 10:00:19,463 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 10:00:19,463 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:19,463 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 10:00:19,464 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,42981,1675936817383, sessionid=0x100388647510000, setting cluster-up flag (Was=false) 2023-02-09 10:00:19,478 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:19,485 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-09 10:00:19,486 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,42981,1675936817383 2023-02-09 10:00:19,491 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:19,497 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-09 10:00:19,498 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,42981,1675936817383 2023-02-09 10:00:19,501 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/.hbase-snapshot/.tmp 2023-02-09 10:00:19,527 INFO [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(952): ClusterId : e9e9143f-54f9-4717-9ab8-f25d43186326 2023-02-09 10:00:19,527 INFO [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(952): ClusterId : e9e9143f-54f9-4717-9ab8-f25d43186326 2023-02-09 10:00:19,527 INFO [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer(952): ClusterId : e9e9143f-54f9-4717-9ab8-f25d43186326 2023-02-09 10:00:19,532 DEBUG [RS:0;jenkins-hbase4:33883] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 10:00:19,532 DEBUG [RS:2;jenkins-hbase4:36117] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 10:00:19,532 DEBUG [RS:1;jenkins-hbase4:39893] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 10:00:19,540 DEBUG [RS:0;jenkins-hbase4:33883] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 10:00:19,540 DEBUG [RS:1;jenkins-hbase4:39893] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 10:00:19,540 DEBUG [RS:2;jenkins-hbase4:36117] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 10:00:19,540 DEBUG [RS:1;jenkins-hbase4:39893] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 10:00:19,540 DEBUG [RS:0;jenkins-hbase4:33883] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 10:00:19,540 DEBUG [RS:2;jenkins-hbase4:36117] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 10:00:19,543 DEBUG [RS:2;jenkins-hbase4:36117] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 10:00:19,543 DEBUG [RS:0;jenkins-hbase4:33883] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 10:00:19,543 DEBUG [RS:1;jenkins-hbase4:39893] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 10:00:19,545 DEBUG [RS:2;jenkins-hbase4:36117] zookeeper.ReadOnlyZKClient(139): Connect 0x617b1bd8 to 127.0.0.1:51004 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 10:00:19,546 DEBUG [RS:0;jenkins-hbase4:33883] zookeeper.ReadOnlyZKClient(139): Connect 0x27e6ec7d to 127.0.0.1:51004 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 10:00:19,546 DEBUG [RS:1;jenkins-hbase4:39893] zookeeper.ReadOnlyZKClient(139): Connect 0x45546556 to 127.0.0.1:51004 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 10:00:19,557 DEBUG [RS:2;jenkins-hbase4:36117] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6fbce987, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 10:00:19,557 DEBUG [RS:0;jenkins-hbase4:33883] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@292106f1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 10:00:19,557 DEBUG [RS:1;jenkins-hbase4:39893] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4b93938c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 10:00:19,558 DEBUG [RS:0;jenkins-hbase4:33883] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d1e870d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 10:00:19,558 DEBUG [RS:2;jenkins-hbase4:36117] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b8370a8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 10:00:19,558 DEBUG [RS:1;jenkins-hbase4:39893] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4fac0e26, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 10:00:19,585 DEBUG [RS:1;jenkins-hbase4:39893] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:39893 2023-02-09 10:00:19,585 DEBUG [RS:2;jenkins-hbase4:36117] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:36117 2023-02-09 10:00:19,585 DEBUG [RS:0;jenkins-hbase4:33883] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:33883 2023-02-09 10:00:19,590 INFO [RS:2;jenkins-hbase4:36117] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 10:00:19,591 INFO [RS:1;jenkins-hbase4:39893] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 10:00:19,590 INFO [RS:0;jenkins-hbase4:33883] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 10:00:19,591 INFO [RS:1;jenkins-hbase4:39893] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 10:00:19,591 INFO [RS:2;jenkins-hbase4:36117] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 10:00:19,591 DEBUG [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 10:00:19,591 INFO [RS:0;jenkins-hbase4:33883] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 10:00:19,591 DEBUG [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 10:00:19,592 DEBUG [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 10:00:19,594 INFO [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,42981,1675936817383 with isa=jenkins-hbase4.apache.org/172.31.14.131:36117, startcode=1675936818606 2023-02-09 10:00:19,594 INFO [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,42981,1675936817383 with isa=jenkins-hbase4.apache.org/172.31.14.131:39893, startcode=1675936818573 2023-02-09 10:00:19,595 INFO [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,42981,1675936817383 with isa=jenkins-hbase4.apache.org/172.31.14.131:33883, startcode=1675936818530 2023-02-09 10:00:19,612 DEBUG [RS:0;jenkins-hbase4:33883] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 10:00:19,612 DEBUG [RS:1;jenkins-hbase4:39893] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 10:00:19,612 DEBUG [RS:2;jenkins-hbase4:36117] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 10:00:19,631 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-09 10:00:19,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 10:00:19,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 10:00:19,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 10:00:19,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 10:00:19,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-09 10:00:19,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 10:00:19,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675936849646 2023-02-09 10:00:19,649 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-09 10:00:19,653 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 10:00:19,653 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-09 10:00:19,659 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 10:00:19,661 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-09 10:00:19,670 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55413, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 10:00:19,671 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-09 10:00:19,670 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47923, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 10:00:19,670 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56315, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 10:00:19,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-09 10:00:19,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-09 10:00:19,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-09 10:00:19,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-09 10:00:19,678 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-09 10:00:19,678 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-09 10:00:19,680 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-09 10:00:19,681 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-09 10:00:19,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42981] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-09 10:00:19,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42981] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-09 10:00:19,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42981] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-09 10:00:19,683 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675936819683,5,FailOnTimeoutGroup] 2023-02-09 10:00:19,684 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675936819683,5,FailOnTimeoutGroup] 2023-02-09 10:00:19,684 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,684 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-09 10:00:19,685 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,686 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,713 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 10:00:19,714 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 10:00:19,714 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6 2023-02-09 10:00:19,724 DEBUG [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(2829): Master is not running yet 2023-02-09 10:00:19,724 DEBUG [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer(2829): Master is not running yet 2023-02-09 10:00:19,724 WARN [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-09 10:00:19,724 DEBUG [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(2829): Master is not running yet 2023-02-09 10:00:19,724 WARN [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-09 10:00:19,724 WARN [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-09 10:00:19,740 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 10:00:19,743 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 10:00:19,745 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/info 2023-02-09 10:00:19,746 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 10:00:19,747 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:19,747 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 10:00:19,749 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/rep_barrier 2023-02-09 10:00:19,750 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 10:00:19,751 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:19,751 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 10:00:19,753 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/table 2023-02-09 10:00:19,753 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 10:00:19,754 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:19,756 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740 2023-02-09 10:00:19,757 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740 2023-02-09 10:00:19,760 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 10:00:19,762 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 10:00:19,766 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 10:00:19,767 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74797025, jitterRate=0.11456252634525299}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 10:00:19,767 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675936819740Initializing all the Stores at 1675936819742 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936819742Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675936819742Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936819742Cleaning up temporary data from old regions at 1675936819759 (+17 ms)Cleaning up detritus from prior splits at 1675936819759Region opened successfully at 1675936819767 (+8 ms) 2023-02-09 10:00:19,767 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 10:00:19,767 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 10:00:19,767 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 10:00:19,767 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 10:00:19,767 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 10:00:19,768 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 10:00:19,768 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675936819767Disabling compacts and flushes for region at 1675936819767Disabling writes for close at 1675936819767Writing region close event to WAL at 1675936819768 (+1 ms)Closed at 1675936819768 2023-02-09 10:00:19,774 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 10:00:19,774 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-09 10:00:19,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-09 10:00:19,795 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-09 10:00:19,798 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-09 10:00:19,825 INFO [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,42981,1675936817383 with isa=jenkins-hbase4.apache.org/172.31.14.131:36117, startcode=1675936818606 2023-02-09 10:00:19,825 INFO [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,42981,1675936817383 with isa=jenkins-hbase4.apache.org/172.31.14.131:39893, startcode=1675936818573 2023-02-09 10:00:19,825 INFO [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,42981,1675936817383 with isa=jenkins-hbase4.apache.org/172.31.14.131:33883, startcode=1675936818530 2023-02-09 10:00:19,831 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42981] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33883,1675936818530 2023-02-09 10:00:19,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42981] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,39893,1675936818573 2023-02-09 10:00:19,832 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42981] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36117,1675936818606 2023-02-09 10:00:19,838 DEBUG [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6 2023-02-09 10:00:19,838 DEBUG [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6 2023-02-09 10:00:19,838 DEBUG [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6 2023-02-09 10:00:19,838 DEBUG [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:34347 2023-02-09 10:00:19,838 DEBUG [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:34347 2023-02-09 10:00:19,839 DEBUG [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 10:00:19,838 DEBUG [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:34347 2023-02-09 10:00:19,839 DEBUG [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 10:00:19,839 DEBUG [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 10:00:19,847 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:19,847 DEBUG [RS:1;jenkins-hbase4:39893] zookeeper.ZKUtil(162): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39893,1675936818573 2023-02-09 10:00:19,847 DEBUG [RS:0;jenkins-hbase4:33883] zookeeper.ZKUtil(162): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33883,1675936818530 2023-02-09 10:00:19,847 WARN [RS:1;jenkins-hbase4:39893] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 10:00:19,847 WARN [RS:0;jenkins-hbase4:33883] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 10:00:19,848 INFO [RS:1;jenkins-hbase4:39893] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 10:00:19,847 DEBUG [RS:2;jenkins-hbase4:36117] zookeeper.ZKUtil(162): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36117,1675936818606 2023-02-09 10:00:19,848 DEBUG [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(1947): logDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/WALs/jenkins-hbase4.apache.org,39893,1675936818573 2023-02-09 10:00:19,848 INFO [RS:0;jenkins-hbase4:33883] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 10:00:19,848 WARN [RS:2;jenkins-hbase4:36117] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 10:00:19,849 DEBUG [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer(1947): logDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/WALs/jenkins-hbase4.apache.org,33883,1675936818530 2023-02-09 10:00:19,849 INFO [RS:2;jenkins-hbase4:36117] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 10:00:19,849 DEBUG [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(1947): logDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/WALs/jenkins-hbase4.apache.org,36117,1675936818606 2023-02-09 10:00:19,852 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33883,1675936818530] 2023-02-09 10:00:19,853 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36117,1675936818606] 2023-02-09 10:00:19,853 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39893,1675936818573] 2023-02-09 10:00:19,862 DEBUG [RS:1;jenkins-hbase4:39893] zookeeper.ZKUtil(162): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33883,1675936818530 2023-02-09 10:00:19,863 DEBUG [RS:0;jenkins-hbase4:33883] zookeeper.ZKUtil(162): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33883,1675936818530 2023-02-09 10:00:19,862 DEBUG [RS:2;jenkins-hbase4:36117] zookeeper.ZKUtil(162): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33883,1675936818530 2023-02-09 10:00:19,863 DEBUG [RS:1;jenkins-hbase4:39893] zookeeper.ZKUtil(162): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36117,1675936818606 2023-02-09 10:00:19,863 DEBUG [RS:0;jenkins-hbase4:33883] zookeeper.ZKUtil(162): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36117,1675936818606 2023-02-09 10:00:19,863 DEBUG [RS:2;jenkins-hbase4:36117] zookeeper.ZKUtil(162): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36117,1675936818606 2023-02-09 10:00:19,863 DEBUG [RS:1;jenkins-hbase4:39893] zookeeper.ZKUtil(162): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39893,1675936818573 2023-02-09 10:00:19,864 DEBUG [RS:0;jenkins-hbase4:33883] zookeeper.ZKUtil(162): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39893,1675936818573 2023-02-09 10:00:19,864 DEBUG [RS:2;jenkins-hbase4:36117] zookeeper.ZKUtil(162): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39893,1675936818573 2023-02-09 10:00:19,875 DEBUG [RS:2;jenkins-hbase4:36117] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 10:00:19,875 DEBUG [RS:0;jenkins-hbase4:33883] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 10:00:19,875 DEBUG [RS:1;jenkins-hbase4:39893] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 10:00:19,884 INFO [RS:0;jenkins-hbase4:33883] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 10:00:19,884 INFO [RS:1;jenkins-hbase4:39893] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 10:00:19,884 INFO [RS:2;jenkins-hbase4:36117] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 10:00:19,906 INFO [RS:0;jenkins-hbase4:33883] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 10:00:19,906 INFO [RS:1;jenkins-hbase4:39893] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 10:00:19,906 INFO [RS:2;jenkins-hbase4:36117] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 10:00:19,911 INFO [RS:0;jenkins-hbase4:33883] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 10:00:19,911 INFO [RS:1;jenkins-hbase4:39893] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 10:00:19,912 INFO [RS:0;jenkins-hbase4:33883] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,911 INFO [RS:2;jenkins-hbase4:36117] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 10:00:19,912 INFO [RS:1;jenkins-hbase4:39893] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,912 INFO [RS:2;jenkins-hbase4:36117] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,913 INFO [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 10:00:19,913 INFO [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 10:00:19,913 INFO [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 10:00:19,920 INFO [RS:0;jenkins-hbase4:33883] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,920 INFO [RS:2;jenkins-hbase4:36117] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,920 INFO [RS:1;jenkins-hbase4:39893] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,920 DEBUG [RS:0;jenkins-hbase4:33883] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,921 DEBUG [RS:2;jenkins-hbase4:36117] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,921 DEBUG [RS:0;jenkins-hbase4:33883] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,921 DEBUG [RS:2;jenkins-hbase4:36117] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,921 DEBUG [RS:0;jenkins-hbase4:33883] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,921 DEBUG [RS:1;jenkins-hbase4:39893] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,921 DEBUG [RS:2;jenkins-hbase4:36117] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,921 DEBUG [RS:1;jenkins-hbase4:39893] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,921 DEBUG [RS:2;jenkins-hbase4:36117] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:1;jenkins-hbase4:39893] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:2;jenkins-hbase4:36117] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:1;jenkins-hbase4:39893] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:2;jenkins-hbase4:36117] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 10:00:19,922 DEBUG [RS:1;jenkins-hbase4:39893] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:2;jenkins-hbase4:36117] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:1;jenkins-hbase4:39893] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 10:00:19,921 DEBUG [RS:0;jenkins-hbase4:33883] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:1;jenkins-hbase4:39893] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:0;jenkins-hbase4:33883] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:1;jenkins-hbase4:39893] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:0;jenkins-hbase4:33883] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 10:00:19,922 DEBUG [RS:1;jenkins-hbase4:39893] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:0;jenkins-hbase4:33883] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:1;jenkins-hbase4:39893] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:0;jenkins-hbase4:33883] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:2;jenkins-hbase4:36117] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,922 DEBUG [RS:0;jenkins-hbase4:33883] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,923 DEBUG [RS:2;jenkins-hbase4:36117] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,923 DEBUG [RS:0;jenkins-hbase4:33883] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,923 DEBUG [RS:2;jenkins-hbase4:36117] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:19,924 INFO [RS:1;jenkins-hbase4:39893] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,924 INFO [RS:1;jenkins-hbase4:39893] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,926 INFO [RS:0;jenkins-hbase4:33883] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,926 INFO [RS:1;jenkins-hbase4:39893] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,926 INFO [RS:0;jenkins-hbase4:33883] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,926 INFO [RS:2;jenkins-hbase4:36117] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,926 INFO [RS:0;jenkins-hbase4:33883] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,926 INFO [RS:2;jenkins-hbase4:36117] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,927 INFO [RS:2;jenkins-hbase4:36117] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,944 INFO [RS:1;jenkins-hbase4:39893] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 10:00:19,944 INFO [RS:0;jenkins-hbase4:33883] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 10:00:19,944 INFO [RS:2;jenkins-hbase4:36117] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 10:00:19,947 INFO [RS:1;jenkins-hbase4:39893] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39893,1675936818573-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,947 INFO [RS:0;jenkins-hbase4:33883] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33883,1675936818530-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,947 INFO [RS:2;jenkins-hbase4:36117] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36117,1675936818606-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:19,950 DEBUG [jenkins-hbase4:42981] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-09 10:00:19,955 DEBUG [jenkins-hbase4:42981] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 10:00:19,961 DEBUG [jenkins-hbase4:42981] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 10:00:19,962 DEBUG [jenkins-hbase4:42981] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 10:00:19,962 DEBUG [jenkins-hbase4:42981] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 10:00:19,962 DEBUG [jenkins-hbase4:42981] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 10:00:19,965 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,39893,1675936818573, state=OPENING 2023-02-09 10:00:19,967 INFO [RS:1;jenkins-hbase4:39893] regionserver.Replication(203): jenkins-hbase4.apache.org,39893,1675936818573 started 2023-02-09 10:00:19,968 INFO [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,39893,1675936818573, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39893, sessionid=0x100388647510002 2023-02-09 10:00:19,968 DEBUG [RS:1;jenkins-hbase4:39893] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 10:00:19,968 DEBUG [RS:1;jenkins-hbase4:39893] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39893,1675936818573 2023-02-09 10:00:19,968 DEBUG [RS:1;jenkins-hbase4:39893] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39893,1675936818573' 2023-02-09 10:00:19,968 DEBUG [RS:1;jenkins-hbase4:39893] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 10:00:19,969 DEBUG [RS:1;jenkins-hbase4:39893] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 10:00:19,970 DEBUG [RS:1;jenkins-hbase4:39893] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 10:00:19,970 DEBUG [RS:1;jenkins-hbase4:39893] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 10:00:19,970 DEBUG [RS:1;jenkins-hbase4:39893] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39893,1675936818573 2023-02-09 10:00:19,970 DEBUG [RS:1;jenkins-hbase4:39893] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39893,1675936818573' 2023-02-09 10:00:19,970 DEBUG [RS:1;jenkins-hbase4:39893] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 10:00:19,970 DEBUG [RS:1;jenkins-hbase4:39893] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 10:00:19,971 DEBUG [RS:1;jenkins-hbase4:39893] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 10:00:19,971 INFO [RS:1;jenkins-hbase4:39893] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 10:00:19,971 INFO [RS:1;jenkins-hbase4:39893] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 10:00:19,972 INFO [RS:0;jenkins-hbase4:33883] regionserver.Replication(203): jenkins-hbase4.apache.org,33883,1675936818530 started 2023-02-09 10:00:19,972 INFO [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,33883,1675936818530, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33883, sessionid=0x100388647510001 2023-02-09 10:00:19,974 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-09 10:00:19,976 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:19,977 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 10:00:19,984 INFO [RS:2;jenkins-hbase4:36117] regionserver.Replication(203): jenkins-hbase4.apache.org,36117,1675936818606 started 2023-02-09 10:00:19,984 DEBUG [RS:0;jenkins-hbase4:33883] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 10:00:19,984 INFO [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,36117,1675936818606, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36117, sessionid=0x100388647510003 2023-02-09 10:00:19,985 DEBUG [RS:0;jenkins-hbase4:33883] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33883,1675936818530 2023-02-09 10:00:19,985 DEBUG [RS:2;jenkins-hbase4:36117] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 10:00:19,986 DEBUG [RS:2;jenkins-hbase4:36117] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36117,1675936818606 2023-02-09 10:00:19,986 DEBUG [RS:2;jenkins-hbase4:36117] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36117,1675936818606' 2023-02-09 10:00:19,986 DEBUG [RS:2;jenkins-hbase4:36117] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 10:00:19,985 DEBUG [RS:0;jenkins-hbase4:33883] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33883,1675936818530' 2023-02-09 10:00:19,986 DEBUG [RS:0;jenkins-hbase4:33883] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 10:00:19,987 DEBUG [RS:0;jenkins-hbase4:33883] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 10:00:19,987 DEBUG [RS:2;jenkins-hbase4:36117] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 10:00:19,987 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,39893,1675936818573}] 2023-02-09 10:00:19,987 DEBUG [RS:0;jenkins-hbase4:33883] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 10:00:19,987 DEBUG [RS:2;jenkins-hbase4:36117] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 10:00:19,987 DEBUG [RS:0;jenkins-hbase4:33883] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 10:00:19,988 DEBUG [RS:2;jenkins-hbase4:36117] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 10:00:19,988 DEBUG [RS:0;jenkins-hbase4:33883] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33883,1675936818530 2023-02-09 10:00:19,988 DEBUG [RS:2;jenkins-hbase4:36117] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36117,1675936818606 2023-02-09 10:00:19,988 DEBUG [RS:2;jenkins-hbase4:36117] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36117,1675936818606' 2023-02-09 10:00:19,988 DEBUG [RS:0;jenkins-hbase4:33883] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33883,1675936818530' 2023-02-09 10:00:19,988 DEBUG [RS:0;jenkins-hbase4:33883] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 10:00:19,988 DEBUG [RS:2;jenkins-hbase4:36117] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 10:00:19,988 DEBUG [RS:0;jenkins-hbase4:33883] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 10:00:19,988 DEBUG [RS:2;jenkins-hbase4:36117] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 10:00:19,989 DEBUG [RS:2;jenkins-hbase4:36117] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 10:00:19,989 DEBUG [RS:0;jenkins-hbase4:33883] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 10:00:19,989 INFO [RS:2;jenkins-hbase4:36117] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 10:00:19,989 INFO [RS:2;jenkins-hbase4:36117] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 10:00:19,989 INFO [RS:0;jenkins-hbase4:33883] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 10:00:19,989 INFO [RS:0;jenkins-hbase4:33883] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 10:00:20,082 INFO [RS:1;jenkins-hbase4:39893] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39893%2C1675936818573, suffix=, logDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/WALs/jenkins-hbase4.apache.org,39893,1675936818573, archiveDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/oldWALs, maxLogs=32 2023-02-09 10:00:20,092 INFO [RS:0;jenkins-hbase4:33883] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33883%2C1675936818530, suffix=, logDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/WALs/jenkins-hbase4.apache.org,33883,1675936818530, archiveDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/oldWALs, maxLogs=32 2023-02-09 10:00:20,092 INFO [RS:2;jenkins-hbase4:36117] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36117%2C1675936818606, suffix=, logDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/WALs/jenkins-hbase4.apache.org,36117,1675936818606, archiveDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/oldWALs, maxLogs=32 2023-02-09 10:00:20,105 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37805,DS-74c4397f-ec33-44d3-8a28-a1b555a8ddbd,DISK] 2023-02-09 10:00:20,105 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38331,DS-c5f29a4c-f95b-430e-9505-66d38f6639a3,DISK] 2023-02-09 10:00:20,105 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43439,DS-8f7eed21-4181-4bee-9bca-b4550cc10328,DISK] 2023-02-09 10:00:20,120 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43439,DS-8f7eed21-4181-4bee-9bca-b4550cc10328,DISK] 2023-02-09 10:00:20,121 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37805,DS-74c4397f-ec33-44d3-8a28-a1b555a8ddbd,DISK] 2023-02-09 10:00:20,121 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38331,DS-c5f29a4c-f95b-430e-9505-66d38f6639a3,DISK] 2023-02-09 10:00:20,122 INFO [RS:1;jenkins-hbase4:39893] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/WALs/jenkins-hbase4.apache.org,39893,1675936818573/jenkins-hbase4.apache.org%2C39893%2C1675936818573.1675936820085 2023-02-09 10:00:20,123 DEBUG [RS:1;jenkins-hbase4:39893] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37805,DS-74c4397f-ec33-44d3-8a28-a1b555a8ddbd,DISK], DatanodeInfoWithStorage[127.0.0.1:38331,DS-c5f29a4c-f95b-430e-9505-66d38f6639a3,DISK], DatanodeInfoWithStorage[127.0.0.1:43439,DS-8f7eed21-4181-4bee-9bca-b4550cc10328,DISK]] 2023-02-09 10:00:20,126 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37805,DS-74c4397f-ec33-44d3-8a28-a1b555a8ddbd,DISK] 2023-02-09 10:00:20,127 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38331,DS-c5f29a4c-f95b-430e-9505-66d38f6639a3,DISK] 2023-02-09 10:00:20,127 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43439,DS-8f7eed21-4181-4bee-9bca-b4550cc10328,DISK] 2023-02-09 10:00:20,134 INFO [RS:2;jenkins-hbase4:36117] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/WALs/jenkins-hbase4.apache.org,36117,1675936818606/jenkins-hbase4.apache.org%2C36117%2C1675936818606.1675936820094 2023-02-09 10:00:20,134 DEBUG [RS:2;jenkins-hbase4:36117] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43439,DS-8f7eed21-4181-4bee-9bca-b4550cc10328,DISK], DatanodeInfoWithStorage[127.0.0.1:37805,DS-74c4397f-ec33-44d3-8a28-a1b555a8ddbd,DISK], DatanodeInfoWithStorage[127.0.0.1:38331,DS-c5f29a4c-f95b-430e-9505-66d38f6639a3,DISK]] 2023-02-09 10:00:20,137 INFO [RS:0;jenkins-hbase4:33883] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/WALs/jenkins-hbase4.apache.org,33883,1675936818530/jenkins-hbase4.apache.org%2C33883%2C1675936818530.1675936820094 2023-02-09 10:00:20,140 DEBUG [RS:0;jenkins-hbase4:33883] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37805,DS-74c4397f-ec33-44d3-8a28-a1b555a8ddbd,DISK], DatanodeInfoWithStorage[127.0.0.1:38331,DS-c5f29a4c-f95b-430e-9505-66d38f6639a3,DISK], DatanodeInfoWithStorage[127.0.0.1:43439,DS-8f7eed21-4181-4bee-9bca-b4550cc10328,DISK]] 2023-02-09 10:00:20,175 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,39893,1675936818573 2023-02-09 10:00:20,178 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 10:00:20,181 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56666, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 10:00:20,196 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-09 10:00:20,196 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 10:00:20,199 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39893%2C1675936818573.meta, suffix=.meta, logDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/WALs/jenkins-hbase4.apache.org,39893,1675936818573, archiveDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/oldWALs, maxLogs=32 2023-02-09 10:00:20,220 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43439,DS-8f7eed21-4181-4bee-9bca-b4550cc10328,DISK] 2023-02-09 10:00:20,221 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37805,DS-74c4397f-ec33-44d3-8a28-a1b555a8ddbd,DISK] 2023-02-09 10:00:20,222 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38331,DS-c5f29a4c-f95b-430e-9505-66d38f6639a3,DISK] 2023-02-09 10:00:20,227 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/WALs/jenkins-hbase4.apache.org,39893,1675936818573/jenkins-hbase4.apache.org%2C39893%2C1675936818573.meta.1675936820201.meta 2023-02-09 10:00:20,227 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43439,DS-8f7eed21-4181-4bee-9bca-b4550cc10328,DISK], DatanodeInfoWithStorage[127.0.0.1:37805,DS-74c4397f-ec33-44d3-8a28-a1b555a8ddbd,DISK], DatanodeInfoWithStorage[127.0.0.1:38331,DS-c5f29a4c-f95b-430e-9505-66d38f6639a3,DISK]] 2023-02-09 10:00:20,228 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-09 10:00:20,229 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-09 10:00:20,246 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-09 10:00:20,253 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-09 10:00:20,260 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-09 10:00:20,260 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 10:00:20,260 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-09 10:00:20,260 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-09 10:00:20,263 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 10:00:20,265 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/info 2023-02-09 10:00:20,265 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/info 2023-02-09 10:00:20,265 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 10:00:20,266 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:20,266 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 10:00:20,268 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/rep_barrier 2023-02-09 10:00:20,268 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/rep_barrier 2023-02-09 10:00:20,268 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 10:00:20,269 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:20,269 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 10:00:20,271 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/table 2023-02-09 10:00:20,271 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/table 2023-02-09 10:00:20,271 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 10:00:20,272 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:20,274 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740 2023-02-09 10:00:20,277 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740 2023-02-09 10:00:20,280 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 10:00:20,282 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 10:00:20,283 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63736436, jitterRate=-0.050253093242645264}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 10:00:20,284 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675936820260Writing region info on filesystem at 1675936820260Initializing all the Stores at 1675936820262 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936820262Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675936820263 (+1 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936820263Cleaning up temporary data from old regions at 1675936820278 (+15 ms)Cleaning up detritus from prior splits at 1675936820279 (+1 ms)Running coprocessor post-open hooks at 1675936820284 (+5 ms)Region opened successfully at 1675936820284 2023-02-09 10:00:20,293 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675936820167 2023-02-09 10:00:20,312 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-09 10:00:20,312 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-09 10:00:20,313 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,39893,1675936818573, state=OPEN 2023-02-09 10:00:20,316 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-09 10:00:20,316 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 10:00:20,320 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-09 10:00:20,320 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,39893,1675936818573 in 329 msec 2023-02-09 10:00:20,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-09 10:00:20,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 539 msec 2023-02-09 10:00:20,331 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 774 msec 2023-02-09 10:00:20,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675936818649, completionTime=-1 2023-02-09 10:00:20,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-09 10:00:20,331 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 10:00:20,397 DEBUG [hconnection-0x4f0e2163-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 10:00:20,400 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56678, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 10:00:20,416 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 10:00:20,416 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675936880416 2023-02-09 10:00:20,417 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675936940417 2023-02-09 10:00:20,417 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 85 msec 2023-02-09 10:00:20,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42981,1675936817383-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:20,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42981,1675936817383-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:20,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42981,1675936817383-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:20,442 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:42981, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:20,442 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:20,452 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-09 10:00:20,462 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-09 10:00:20,463 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 10:00:20,473 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-09 10:00:20,475 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 10:00:20,478 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 10:00:20,500 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/.tmp/data/hbase/namespace/1ce9ec2f2ef957d6f93bbe21c8dc8c70 2023-02-09 10:00:20,503 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/.tmp/data/hbase/namespace/1ce9ec2f2ef957d6f93bbe21c8dc8c70 empty. 2023-02-09 10:00:20,504 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/.tmp/data/hbase/namespace/1ce9ec2f2ef957d6f93bbe21c8dc8c70 2023-02-09 10:00:20,504 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-09 10:00:20,544 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-09 10:00:20,546 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1ce9ec2f2ef957d6f93bbe21c8dc8c70, NAME => 'hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/.tmp 2023-02-09 10:00:20,567 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 10:00:20,568 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 1ce9ec2f2ef957d6f93bbe21c8dc8c70, disabling compactions & flushes 2023-02-09 10:00:20,568 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. 2023-02-09 10:00:20,568 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. 2023-02-09 10:00:20,568 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. after waiting 0 ms 2023-02-09 10:00:20,568 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. 2023-02-09 10:00:20,568 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. 2023-02-09 10:00:20,568 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 1ce9ec2f2ef957d6f93bbe21c8dc8c70: Waiting for close lock at 1675936820568Disabling compacts and flushes for region at 1675936820568Disabling writes for close at 1675936820568Writing region close event to WAL at 1675936820568Closed at 1675936820568 2023-02-09 10:00:20,572 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 10:00:20,588 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675936820575"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936820575"}]},"ts":"1675936820575"} 2023-02-09 10:00:20,617 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 10:00:20,619 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 10:00:20,623 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936820619"}]},"ts":"1675936820619"} 2023-02-09 10:00:20,628 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-09 10:00:20,634 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 10:00:20,635 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 10:00:20,635 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 10:00:20,635 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 10:00:20,635 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 10:00:20,638 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=1ce9ec2f2ef957d6f93bbe21c8dc8c70, ASSIGN}] 2023-02-09 10:00:20,641 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=1ce9ec2f2ef957d6f93bbe21c8dc8c70, ASSIGN 2023-02-09 10:00:20,643 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=1ce9ec2f2ef957d6f93bbe21c8dc8c70, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36117,1675936818606; forceNewPlan=false, retain=false 2023-02-09 10:00:20,795 INFO [jenkins-hbase4:42981] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 10:00:20,796 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=1ce9ec2f2ef957d6f93bbe21c8dc8c70, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36117,1675936818606 2023-02-09 10:00:20,796 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675936820796"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936820796"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936820796"}]},"ts":"1675936820796"} 2023-02-09 10:00:20,800 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 1ce9ec2f2ef957d6f93bbe21c8dc8c70, server=jenkins-hbase4.apache.org,36117,1675936818606}] 2023-02-09 10:00:20,954 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36117,1675936818606 2023-02-09 10:00:20,955 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 10:00:20,959 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36368, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 10:00:20,965 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. 2023-02-09 10:00:20,966 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1ce9ec2f2ef957d6f93bbe21c8dc8c70, NAME => 'hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70.', STARTKEY => '', ENDKEY => ''} 2023-02-09 10:00:20,967 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 1ce9ec2f2ef957d6f93bbe21c8dc8c70 2023-02-09 10:00:20,967 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 10:00:20,967 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1ce9ec2f2ef957d6f93bbe21c8dc8c70 2023-02-09 10:00:20,967 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1ce9ec2f2ef957d6f93bbe21c8dc8c70 2023-02-09 10:00:20,969 INFO [StoreOpener-1ce9ec2f2ef957d6f93bbe21c8dc8c70-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1ce9ec2f2ef957d6f93bbe21c8dc8c70 2023-02-09 10:00:20,971 DEBUG [StoreOpener-1ce9ec2f2ef957d6f93bbe21c8dc8c70-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/namespace/1ce9ec2f2ef957d6f93bbe21c8dc8c70/info 2023-02-09 10:00:20,971 DEBUG [StoreOpener-1ce9ec2f2ef957d6f93bbe21c8dc8c70-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/namespace/1ce9ec2f2ef957d6f93bbe21c8dc8c70/info 2023-02-09 10:00:20,972 INFO [StoreOpener-1ce9ec2f2ef957d6f93bbe21c8dc8c70-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ce9ec2f2ef957d6f93bbe21c8dc8c70 columnFamilyName info 2023-02-09 10:00:20,973 INFO [StoreOpener-1ce9ec2f2ef957d6f93bbe21c8dc8c70-1] regionserver.HStore(310): Store=1ce9ec2f2ef957d6f93bbe21c8dc8c70/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:20,975 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/namespace/1ce9ec2f2ef957d6f93bbe21c8dc8c70 2023-02-09 10:00:20,976 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/namespace/1ce9ec2f2ef957d6f93bbe21c8dc8c70 2023-02-09 10:00:20,979 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1ce9ec2f2ef957d6f93bbe21c8dc8c70 2023-02-09 10:00:20,983 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/namespace/1ce9ec2f2ef957d6f93bbe21c8dc8c70/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 10:00:20,984 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1ce9ec2f2ef957d6f93bbe21c8dc8c70; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59491614, jitterRate=-0.11350587010383606}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 10:00:20,984 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1ce9ec2f2ef957d6f93bbe21c8dc8c70: Running coprocessor pre-open hook at 1675936820967Writing region info on filesystem at 1675936820967Initializing all the Stores at 1675936820969 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936820969Cleaning up temporary data from old regions at 1675936820976 (+7 ms)Cleaning up detritus from prior splits at 1675936820977 (+1 ms)Running coprocessor post-open hooks at 1675936820984 (+7 ms)Region opened successfully at 1675936820984 2023-02-09 10:00:20,986 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70., pid=6, masterSystemTime=1675936820954 2023-02-09 10:00:20,990 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. 2023-02-09 10:00:20,991 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. 2023-02-09 10:00:20,993 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=1ce9ec2f2ef957d6f93bbe21c8dc8c70, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36117,1675936818606 2023-02-09 10:00:20,993 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675936820992"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936820992"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936820992"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936820992"}]},"ts":"1675936820992"} 2023-02-09 10:00:21,001 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-09 10:00:21,001 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 1ce9ec2f2ef957d6f93bbe21c8dc8c70, server=jenkins-hbase4.apache.org,36117,1675936818606 in 198 msec 2023-02-09 10:00:21,005 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-09 10:00:21,006 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=1ce9ec2f2ef957d6f93bbe21c8dc8c70, ASSIGN in 363 msec 2023-02-09 10:00:21,007 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 10:00:21,007 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936821007"}]},"ts":"1675936821007"} 2023-02-09 10:00:21,009 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-09 10:00:21,013 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 10:00:21,015 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 549 msec 2023-02-09 10:00:21,076 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-09 10:00:21,077 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-09 10:00:21,077 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:21,098 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 10:00:21,100 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36380, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 10:00:21,112 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-09 10:00:21,129 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 10:00:21,135 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 29 msec 2023-02-09 10:00:21,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-09 10:00:21,157 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 10:00:21,163 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 17 msec 2023-02-09 10:00:21,172 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-09 10:00:21,175 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-09 10:00:21,175 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.514sec 2023-02-09 10:00:21,178 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-09 10:00:21,180 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-09 10:00:21,180 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-09 10:00:21,181 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42981,1675936817383-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-09 10:00:21,183 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42981,1675936817383-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-09 10:00:21,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-09 10:00:21,233 DEBUG [Listener at localhost/33549] zookeeper.ReadOnlyZKClient(139): Connect 0x24d4d701 to 127.0.0.1:51004 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 10:00:21,242 DEBUG [Listener at localhost/33549] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@69f65fe1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 10:00:21,256 DEBUG [hconnection-0x6274e043-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 10:00:21,269 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56682, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 10:00:21,279 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,42981,1675936817383 2023-02-09 10:00:21,280 DEBUG [Listener at localhost/33549] zookeeper.ReadOnlyZKClient(139): Connect 0x509b497f to 127.0.0.1:51004 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 10:00:21,291 DEBUG [ReadOnlyZKClient-127.0.0.1:51004@0x509b497f] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1bbee375, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 10:00:21,361 DEBUG [Listener at localhost/33549] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 10:00:21,365 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37882, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 10:00:21,366 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33883] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33883,1675936818530' ***** 2023-02-09 10:00:21,366 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33883] regionserver.HRegionServer(2310): STOPPED: Called by admin client org.apache.hadoop.hbase.client.AsyncConnectionImpl@61bce98f 2023-02-09 10:00:21,367 INFO [RS:0;jenkins-hbase4:33883] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 10:00:21,367 INFO [RS:0;jenkins-hbase4:33883] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 10:00:21,367 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 10:00:21,368 INFO [RS:0;jenkins-hbase4:33883] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 10:00:21,368 INFO [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33883,1675936818530 2023-02-09 10:00:21,369 DEBUG [RS:0;jenkins-hbase4:33883] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x27e6ec7d to 127.0.0.1:51004 2023-02-09 10:00:21,369 DEBUG [RS:0;jenkins-hbase4:33883] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:21,369 INFO [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33883,1675936818530; all regions closed. 2023-02-09 10:00:21,373 DEBUG [Listener at localhost/33549] client.ConnectionUtils(586): Start fetching master stub from registry 2023-02-09 10:00:21,375 DEBUG [ReadOnlyZKClient-127.0.0.1:51004@0x509b497f] client.AsyncConnectionImpl(289): The fetched master address is jenkins-hbase4.apache.org,42981,1675936817383 2023-02-09 10:00:21,379 DEBUG [ReadOnlyZKClient-127.0.0.1:51004@0x509b497f] client.ConnectionUtils(594): The fetched master stub is org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$Stub@3dc88106 2023-02-09 10:00:21,386 DEBUG [ReadOnlyZKClient-127.0.0.1:51004@0x509b497f] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 10:00:21,391 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40708, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 10:00:21,391 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42981] master.MasterRpcServices(1601): Client=jenkins//172.31.14.131 stop 2023-02-09 10:00:21,391 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42981] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,42981,1675936817383' ***** 2023-02-09 10:00:21,391 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42981] regionserver.HRegionServer(2310): STOPPED: Stopped by RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42981 2023-02-09 10:00:21,393 DEBUG [M:0;jenkins-hbase4:42981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6067ebe7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 10:00:21,393 INFO [M:0;jenkins-hbase4:42981] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,42981,1675936817383 2023-02-09 10:00:21,395 DEBUG [M:0;jenkins-hbase4:42981] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x041fbeaa to 127.0.0.1:51004 2023-02-09 10:00:21,395 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-09 10:00:21,396 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-09 10:00:21,396 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-09 10:00:21,395 DEBUG [M:0;jenkins-hbase4:42981] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:21,396 INFO [M:0;jenkins-hbase4:42981] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,42981,1675936817383; all regions closed. 2023-02-09 10:00:21,396 DEBUG [M:0;jenkins-hbase4:42981] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:21,397 DEBUG [M:0;jenkins-hbase4:42981] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-09 10:00:21,397 DEBUG [M:0;jenkins-hbase4:42981] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-09 10:00:21,397 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675936819683] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675936819683,5,FailOnTimeoutGroup] 2023-02-09 10:00:21,397 INFO [M:0;jenkins-hbase4:42981] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-09 10:00:21,396 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-09 10:00:21,399 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:21,398 INFO [M:0;jenkins-hbase4:42981] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-09 10:00:21,399 DEBUG [RS:0;jenkins-hbase4:33883] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/oldWALs 2023-02-09 10:00:21,397 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-09 10:00:21,399 INFO [M:0;jenkins-hbase4:42981] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-09 10:00:21,400 DEBUG [M:0;jenkins-hbase4:42981] master.HMaster(1502): Stopping service threads 2023-02-09 10:00:21,400 INFO [M:0;jenkins-hbase4:42981] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-09 10:00:21,399 INFO [RS:0;jenkins-hbase4:33883] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33883%2C1675936818530:(num 1675936820094) 2023-02-09 10:00:21,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 10:00:21,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675936819683] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675936819683,5,FailOnTimeoutGroup] 2023-02-09 10:00:21,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 10:00:21,401 INFO [Listener at localhost/33549] client.AsyncConnectionImpl(207): Connection has been closed by Listener at localhost/33549. 2023-02-09 10:00:21,401 DEBUG [RS:0;jenkins-hbase4:33883] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:21,401 INFO [M:0;jenkins-hbase4:42981] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 10:00:21,402 DEBUG [Listener at localhost/33549] client.AsyncConnectionImpl(232): Call stack: at java.lang.Thread.getStackTrace(Thread.java:1564) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:209) at org.apache.hbase.thirdparty.com.google.common.io.Closeables.close(Closeables.java:79) at org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2.tearDown(TestAsyncClusterAdminApi2.java:75) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2023-02-09 10:00:21,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 10:00:21,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 10:00:21,402 INFO [RS:0;jenkins-hbase4:33883] regionserver.LeaseManager(133): Closed leases 2023-02-09 10:00:21,404 ERROR [M:0;jenkins-hbase4:42981] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] 2023-02-09 10:00:21,404 INFO [M:0;jenkins-hbase4:42981] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 10:00:21,405 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-09 10:00:21,405 INFO [RS:0;jenkins-hbase4:33883] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-09 10:00:21,406 INFO [RS:0;jenkins-hbase4:33883] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 10:00:21,406 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 10:00:21,406 INFO [RS:0;jenkins-hbase4:33883] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 10:00:21,406 DEBUG [M:0;jenkins-hbase4:42981] zookeeper.ZKUtil(398): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-09 10:00:21,406 INFO [RS:0;jenkins-hbase4:33883] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 10:00:21,406 WARN [M:0;jenkins-hbase4:42981] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-09 10:00:21,407 INFO [M:0;jenkins-hbase4:42981] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 10:00:21,407 INFO [M:0;jenkins-hbase4:42981] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-09 10:00:21,408 INFO [RS:0;jenkins-hbase4:33883] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33883 2023-02-09 10:00:21,408 DEBUG [M:0;jenkins-hbase4:42981] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 10:00:21,408 INFO [M:0;jenkins-hbase4:42981] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:21,408 DEBUG [M:0;jenkins-hbase4:42981] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:21,408 DEBUG [M:0;jenkins-hbase4:42981] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 10:00:21,408 DEBUG [M:0;jenkins-hbase4:42981] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:21,409 INFO [M:0;jenkins-hbase4:42981] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-02-09 10:00:21,412 DEBUG [Listener at localhost/33549] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:21,413 DEBUG [Listener at localhost/33549] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x509b497f to 127.0.0.1:51004 2023-02-09 10:00:21,413 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-09 10:00:21,413 DEBUG [Listener at localhost/33549] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x24d4d701 to 127.0.0.1:51004 2023-02-09 10:00:21,413 DEBUG [Listener at localhost/33549] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:21,414 DEBUG [Listener at localhost/33549] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-09 10:00:21,414 INFO [Listener at localhost/33549] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,39893,1675936818573' ***** 2023-02-09 10:00:21,414 INFO [Listener at localhost/33549] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 10:00:21,414 INFO [Listener at localhost/33549] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,36117,1675936818606' ***** 2023-02-09 10:00:21,414 INFO [Listener at localhost/33549] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 10:00:21,415 INFO [RS:2;jenkins-hbase4:36117] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 10:00:21,416 INFO [RS:1;jenkins-hbase4:39893] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 10:00:21,416 INFO [RS:2;jenkins-hbase4:36117] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 10:00:21,416 INFO [RS:1;jenkins-hbase4:39893] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 10:00:21,416 INFO [RS:2;jenkins-hbase4:36117] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 10:00:21,416 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 10:00:21,416 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 10:00:21,416 INFO [RS:1;jenkins-hbase4:39893] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 10:00:21,417 INFO [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(3304): Received CLOSE for 1ce9ec2f2ef957d6f93bbe21c8dc8c70 2023-02-09 10:00:21,417 INFO [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,39893,1675936818573 2023-02-09 10:00:21,417 DEBUG [RS:1;jenkins-hbase4:39893] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x45546556 to 127.0.0.1:51004 2023-02-09 10:00:21,418 DEBUG [RS:1;jenkins-hbase4:39893] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:21,418 INFO [RS:1;jenkins-hbase4:39893] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 10:00:21,418 INFO [RS:1;jenkins-hbase4:39893] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 10:00:21,419 INFO [RS:1;jenkins-hbase4:39893] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 10:00:21,419 INFO [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-09 10:00:21,419 INFO [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,36117,1675936818606 2023-02-09 10:00:21,420 DEBUG [RS:2;jenkins-hbase4:36117] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x617b1bd8 to 127.0.0.1:51004 2023-02-09 10:00:21,420 DEBUG [RS:2;jenkins-hbase4:36117] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:21,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1ce9ec2f2ef957d6f93bbe21c8dc8c70, disabling compactions & flushes 2023-02-09 10:00:21,420 INFO [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-09 10:00:21,420 INFO [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-09 10:00:21,421 DEBUG [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(1479): Online Regions={1ce9ec2f2ef957d6f93bbe21c8dc8c70=hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70.} 2023-02-09 10:00:21,422 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33883,1675936818530 2023-02-09 10:00:21,422 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:21,421 DEBUG [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-09 10:00:21,421 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 10:00:21,422 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 10:00:21,422 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 10:00:21,421 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. 2023-02-09 10:00:21,422 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 10:00:21,422 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:21,422 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33883,1675936818530 2023-02-09 10:00:21,422 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33883,1675936818530 2023-02-09 10:00:21,423 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:21,422 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 10:00:21,422 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. 2023-02-09 10:00:21,423 ERROR [Listener at localhost/33549-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@15b7b3fb rejected from java.util.concurrent.ThreadPoolExecutor@11883595[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:21,423 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-02-09 10:00:21,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. after waiting 0 ms 2023-02-09 10:00:21,423 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:21,423 DEBUG [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(1505): Waiting on 1ce9ec2f2ef957d6f93bbe21c8dc8c70 2023-02-09 10:00:21,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. 2023-02-09 10:00:21,423 ERROR [Listener at localhost/33549-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4d1442c2 rejected from java.util.concurrent.ThreadPoolExecutor@11883595[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:21,423 DEBUG [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 10:00:21,423 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1ce9ec2f2ef957d6f93bbe21c8dc8c70 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-09 10:00:21,455 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 10:00:21,455 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 10:00:21,455 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 10:00:21,509 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/.tmp/info/e2d9a7379e044e1484b87ec8be02cc1f 2023-02-09 10:00:21,509 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/namespace/1ce9ec2f2ef957d6f93bbe21c8dc8c70/.tmp/info/e6e9391f393749d7a764d329888d24e0 2023-02-09 10:00:21,509 INFO [M:0;jenkins-hbase4:42981] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3fc8aa2124d24b75923f766a775cecf9 2023-02-09 10:00:21,553 DEBUG [M:0;jenkins-hbase4:42981] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3fc8aa2124d24b75923f766a775cecf9 as hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3fc8aa2124d24b75923f766a775cecf9 2023-02-09 10:00:21,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/namespace/1ce9ec2f2ef957d6f93bbe21c8dc8c70/.tmp/info/e6e9391f393749d7a764d329888d24e0 as hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/namespace/1ce9ec2f2ef957d6f93bbe21c8dc8c70/info/e6e9391f393749d7a764d329888d24e0 2023-02-09 10:00:21,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/namespace/1ce9ec2f2ef957d6f93bbe21c8dc8c70/info/e6e9391f393749d7a764d329888d24e0, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 10:00:21,566 INFO [M:0;jenkins-hbase4:42981] regionserver.HStore(1080): Added hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3fc8aa2124d24b75923f766a775cecf9, entries=8, sequenceid=66, filesize=6.3 K 2023-02-09 10:00:21,569 INFO [M:0;jenkins-hbase4:42981] regionserver.HRegion(2947): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 160ms, sequenceid=66, compaction requested=false 2023-02-09 10:00:21,569 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 1ce9ec2f2ef957d6f93bbe21c8dc8c70 in 146ms, sequenceid=6, compaction requested=false 2023-02-09 10:00:21,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-09 10:00:21,572 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:21,573 ERROR [Listener at localhost/33549-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@35e0ad4a rejected from java.util.concurrent.ThreadPoolExecutor@11883595[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:21,573 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:33883-0x100388647510001, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:21,573 ERROR [Listener at localhost/33549-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5b48008b rejected from java.util.concurrent.ThreadPoolExecutor@11883595[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:21,573 INFO [RS:0;jenkins-hbase4:33883] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33883,1675936818530; zookeeper connection closed. 2023-02-09 10:00:21,573 INFO [M:0;jenkins-hbase4:42981] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:21,573 DEBUG [M:0;jenkins-hbase4:42981] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675936821408Disabling compacts and flushes for region at 1675936821408Disabling writes for close at 1675936821408Obtaining lock to block concurrent updates at 1675936821409 (+1 ms)Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675936821409Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=24646, getHeapSize=30248, getOffHeapSize=0, getCellsCount=71 at 1675936821425 (+16 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675936821427 (+2 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675936821429 (+2 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675936821460 (+31 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675936821464 (+4 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675936821554 (+90 ms)Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 160ms, sequenceid=66, compaction requested=false at 1675936821569 (+15 ms)Writing region close event to WAL at 1675936821573 (+4 ms)Closed at 1675936821573 2023-02-09 10:00:21,573 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@dc0c723] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@dc0c723 2023-02-09 10:00:21,585 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 10:00:21,585 INFO [M:0;jenkins-hbase4:42981] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-09 10:00:21,586 INFO [M:0;jenkins-hbase4:42981] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42981 2023-02-09 10:00:21,588 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/.tmp/table/40954f0234ee4c44bfebc78b3c9398ab 2023-02-09 10:00:21,589 DEBUG [M:0;jenkins-hbase4:42981] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,42981,1675936817383 already deleted, retry=false 2023-02-09 10:00:21,591 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/namespace/1ce9ec2f2ef957d6f93bbe21c8dc8c70/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-09 10:00:21,592 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. 2023-02-09 10:00:21,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1ce9ec2f2ef957d6f93bbe21c8dc8c70: Waiting for close lock at 1675936821420Running coprocessor pre-close hooks at 1675936821420Disabling compacts and flushes for region at 1675936821420Disabling writes for close at 1675936821423 (+3 ms)Obtaining lock to block concurrent updates at 1675936821424 (+1 ms)Preparing flush snapshotting stores in 1ce9ec2f2ef957d6f93bbe21c8dc8c70 at 1675936821424Finished memstore snapshotting hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1675936821425 (+1 ms)Flushing stores of hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. at 1675936821427 (+2 ms)Flushing 1ce9ec2f2ef957d6f93bbe21c8dc8c70/info: creating writer at 1675936821429 (+2 ms)Flushing 1ce9ec2f2ef957d6f93bbe21c8dc8c70/info: appending metadata at 1675936821460 (+31 ms)Flushing 1ce9ec2f2ef957d6f93bbe21c8dc8c70/info: closing flushed file at 1675936821464 (+4 ms)Flushing 1ce9ec2f2ef957d6f93bbe21c8dc8c70/info: reopening flushed file at 1675936821556 (+92 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 1ce9ec2f2ef957d6f93bbe21c8dc8c70 in 146ms, sequenceid=6, compaction requested=false at 1675936821569 (+13 ms)Writing region close event to WAL at 1675936821577 (+8 ms)Running coprocessor post-close hooks at 1675936821592 (+15 ms)Closed at 1675936821592 2023-02-09 10:00:21,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675936820463.1ce9ec2f2ef957d6f93bbe21c8dc8c70. 2023-02-09 10:00:21,598 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/.tmp/info/e2d9a7379e044e1484b87ec8be02cc1f as hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/info/e2d9a7379e044e1484b87ec8be02cc1f 2023-02-09 10:00:21,607 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/info/e2d9a7379e044e1484b87ec8be02cc1f, entries=10, sequenceid=9, filesize=5.9 K 2023-02-09 10:00:21,609 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/.tmp/table/40954f0234ee4c44bfebc78b3c9398ab as hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/table/40954f0234ee4c44bfebc78b3c9398ab 2023-02-09 10:00:21,618 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/table/40954f0234ee4c44bfebc78b3c9398ab, entries=2, sequenceid=9, filesize=4.7 K 2023-02-09 10:00:21,620 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 197ms, sequenceid=9, compaction requested=false 2023-02-09 10:00:21,620 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-09 10:00:21,623 INFO [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,36117,1675936818606; all regions closed. 2023-02-09 10:00:21,624 DEBUG [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 10:00:21,635 DEBUG [RS:2;jenkins-hbase4:36117] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/oldWALs 2023-02-09 10:00:21,635 INFO [RS:2;jenkins-hbase4:36117] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36117%2C1675936818606:(num 1675936820094) 2023-02-09 10:00:21,635 DEBUG [RS:2;jenkins-hbase4:36117] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:21,635 INFO [RS:2;jenkins-hbase4:36117] regionserver.LeaseManager(133): Closed leases 2023-02-09 10:00:21,636 INFO [RS:2;jenkins-hbase4:36117] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-09 10:00:21,636 INFO [RS:2;jenkins-hbase4:36117] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 10:00:21,636 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 10:00:21,636 INFO [RS:2;jenkins-hbase4:36117] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 10:00:21,636 INFO [RS:2;jenkins-hbase4:36117] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 10:00:21,636 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-02-09 10:00:21,637 INFO [RS:2;jenkins-hbase4:36117] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36117 2023-02-09 10:00:21,640 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 10:00:21,641 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 10:00:21,641 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675936821421Running coprocessor pre-close hooks at 1675936821421Disabling compacts and flushes for region at 1675936821421Disabling writes for close at 1675936821422 (+1 ms)Obtaining lock to block concurrent updates at 1675936821423 (+1 ms)Preparing flush snapshotting stores in 1588230740 at 1675936821423Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=1290, getHeapSize=2912, getOffHeapSize=0, getCellsCount=12 at 1675936821425 (+2 ms)Flushing stores of hbase:meta,,1.1588230740 at 1675936821427 (+2 ms)Flushing 1588230740/info: creating writer at 1675936821429 (+2 ms)Flushing 1588230740/info: appending metadata at 1675936821460 (+31 ms)Flushing 1588230740/info: closing flushed file at 1675936821464 (+4 ms)Flushing 1588230740/table: creating writer at 1675936821551 (+87 ms)Flushing 1588230740/table: appending metadata at 1675936821559 (+8 ms)Flushing 1588230740/table: closing flushed file at 1675936821559Flushing 1588230740/info: reopening flushed file at 1675936821599 (+40 ms)Flushing 1588230740/table: reopening flushed file at 1675936821610 (+11 ms)Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 197ms, sequenceid=9, compaction requested=false at 1675936821620 (+10 ms)Writing region close event to WAL at 1675936821627 (+7 ms)Running coprocessor post-close hooks at 1675936821640 (+13 ms)Closed at 1675936821641 (+1 ms) 2023-02-09 10:00:21,641 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-09 10:00:21,642 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36117,1675936818606 2023-02-09 10:00:21,642 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36117,1675936818606 2023-02-09 10:00:21,824 INFO [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,39893,1675936818573; all regions closed. 2023-02-09 10:00:21,833 DEBUG [RS:1;jenkins-hbase4:39893] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/oldWALs 2023-02-09 10:00:21,833 INFO [RS:1;jenkins-hbase4:39893] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39893%2C1675936818573.meta:.meta(num 1675936820201) 2023-02-09 10:00:21,839 DEBUG [RS:1;jenkins-hbase4:39893] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/oldWALs 2023-02-09 10:00:21,840 INFO [RS:1;jenkins-hbase4:39893] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39893%2C1675936818573:(num 1675936820085) 2023-02-09 10:00:21,840 DEBUG [RS:1;jenkins-hbase4:39893] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:21,840 INFO [RS:1;jenkins-hbase4:39893] regionserver.LeaseManager(133): Closed leases 2023-02-09 10:00:21,840 INFO [RS:1;jenkins-hbase4:39893] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-09 10:00:21,840 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 10:00:21,841 INFO [RS:1;jenkins-hbase4:39893] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39893 2023-02-09 10:00:21,843 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39893,1675936818573 2023-02-09 10:00:21,843 ERROR [Listener at localhost/33549-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4220683 rejected from java.util.concurrent.ThreadPoolExecutor@1d92df[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:22,173 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:22,173 INFO [RS:1;jenkins-hbase4:39893] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,39893,1675936818573; zookeeper connection closed. 2023-02-09 10:00:22,174 ERROR [Listener at localhost/33549-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6012bd35 rejected from java.util.concurrent.ThreadPoolExecutor@1d92df[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:22,174 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@35d755c3] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@35d755c3 2023-02-09 10:00:22,174 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:39893-0x100388647510002, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:22,174 ERROR [Listener at localhost/33549-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1877468d rejected from java.util.concurrent.ThreadPoolExecutor@1d92df[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:22,274 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:22,274 INFO [RS:2;jenkins-hbase4:36117] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,36117,1675936818606; zookeeper connection closed. 2023-02-09 10:00:22,274 ERROR [Listener at localhost/33549-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2ec75a9e rejected from java.util.concurrent.ThreadPoolExecutor@2af8db9e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:22,274 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@325023af] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@325023af 2023-02-09 10:00:22,274 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): regionserver:36117-0x100388647510003, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:22,274 ERROR [Listener at localhost/33549-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4ad81a74 rejected from java.util.concurrent.ThreadPoolExecutor@2af8db9e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:22,275 INFO [Listener at localhost/33549] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-09 10:00:22,374 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:22,374 INFO [M:0;jenkins-hbase4:42981] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,42981,1675936817383; zookeeper connection closed. 2023-02-09 10:00:22,374 ERROR [Listener at localhost/33549-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@25b9b50b rejected from java.util.concurrent.ThreadPoolExecutor@590c7d3f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 24] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:22,374 DEBUG [Listener at localhost/33549-EventThread] zookeeper.ZKWatcher(600): master:42981-0x100388647510000, quorum=127.0.0.1:51004, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:22,374 ERROR [Listener at localhost/33549-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@102e25b5 rejected from java.util.concurrent.ThreadPoolExecutor@590c7d3f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 24] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:22,376 WARN [Listener at localhost/33549] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 10:00:22,379 INFO [Listener at localhost/33549] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 10:00:22,482 WARN [BP-1572928511-172.31.14.131-1675936814228 heartbeating to localhost/127.0.0.1:34347] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-09 10:00:22,482 WARN [BP-1572928511-172.31.14.131-1675936814228 heartbeating to localhost/127.0.0.1:34347] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1572928511-172.31.14.131-1675936814228 (Datanode Uuid 26b6dc11-b58f-4fc6-9b14-326fd99c9d1a) service to localhost/127.0.0.1:34347 2023-02-09 10:00:22,484 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/cluster_1ad255c6-176f-12da-fe0a-3d2eb85088ac/dfs/data/data5/current/BP-1572928511-172.31.14.131-1675936814228] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 10:00:22,484 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/cluster_1ad255c6-176f-12da-fe0a-3d2eb85088ac/dfs/data/data6/current/BP-1572928511-172.31.14.131-1675936814228] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 10:00:22,485 WARN [Listener at localhost/33549] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 10:00:22,489 INFO [Listener at localhost/33549] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 10:00:22,591 WARN [BP-1572928511-172.31.14.131-1675936814228 heartbeating to localhost/127.0.0.1:34347] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-09 10:00:22,591 WARN [BP-1572928511-172.31.14.131-1675936814228 heartbeating to localhost/127.0.0.1:34347] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1572928511-172.31.14.131-1675936814228 (Datanode Uuid a116a5ba-577f-46c2-a944-acb8054cda7f) service to localhost/127.0.0.1:34347 2023-02-09 10:00:22,592 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/cluster_1ad255c6-176f-12da-fe0a-3d2eb85088ac/dfs/data/data3/current/BP-1572928511-172.31.14.131-1675936814228] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 10:00:22,592 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/cluster_1ad255c6-176f-12da-fe0a-3d2eb85088ac/dfs/data/data4/current/BP-1572928511-172.31.14.131-1675936814228] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 10:00:22,593 WARN [Listener at localhost/33549] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 10:00:22,595 INFO [Listener at localhost/33549] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 10:00:22,697 WARN [BP-1572928511-172.31.14.131-1675936814228 heartbeating to localhost/127.0.0.1:34347] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-09 10:00:22,697 WARN [BP-1572928511-172.31.14.131-1675936814228 heartbeating to localhost/127.0.0.1:34347] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1572928511-172.31.14.131-1675936814228 (Datanode Uuid 3e3369a7-c746-44e4-bf58-1e5d694f2bef) service to localhost/127.0.0.1:34347 2023-02-09 10:00:22,698 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/cluster_1ad255c6-176f-12da-fe0a-3d2eb85088ac/dfs/data/data1/current/BP-1572928511-172.31.14.131-1675936814228] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 10:00:22,698 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/cluster_1ad255c6-176f-12da-fe0a-3d2eb85088ac/dfs/data/data2/current/BP-1572928511-172.31.14.131-1675936814228] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 10:00:22,723 INFO [Listener at localhost/33549] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 10:00:22,837 INFO [Listener at localhost/33549] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-09 10:00:22,875 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-02-09 10:00:22,888 INFO [Listener at localhost/33549] hbase.ResourceChecker(175): after: client.TestAsyncClusterAdminApi2#testStop Thread=77 (was 8) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@b6c743b java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:300) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-3-worker-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:34347 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: ForkJoinPool-3-worker-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:34347 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'NameNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:34347 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-3-worker-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/33549 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (985377774) connection to localhost/127.0.0.1:34347 from jenkins.hfs.1 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-3-worker-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RS-EventLoopGroup-5-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (985377774) connection to localhost/127.0.0.1:34347 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-5-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (985377774) connection to localhost/127.0.0.1:34347 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-3-worker-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-5-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (985377774) connection to localhost/127.0.0.1:34347 from jenkins.hfs.2 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: IPC Client (985377774) connection to localhost/127.0.0.1:34347 from jenkins.hfs.0 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:34347 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=508 (was 260) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=292 (was 301), ProcessCount=168 (was 168), AvailableMemoryMB=3359 (was 3935) 2023-02-09 10:00:22,899 INFO [Listener at localhost/33549] hbase.ResourceChecker(147): before: client.TestAsyncClusterAdminApi2#testShutdown Thread=77, OpenFileDescriptor=508, MaxFileDescriptor=60000, SystemLoadAverage=292, ProcessCount=168, AvailableMemoryMB=3359 2023-02-09 10:00:22,899 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-09 10:00:22,900 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/hadoop.log.dir so I do NOT create it in target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab 2023-02-09 10:00:22,900 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3e893ecf-f1ee-0636-e238-e76107adbd34/hadoop.tmp.dir so I do NOT create it in target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab 2023-02-09 10:00:22,900 INFO [Listener at localhost/33549] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/cluster_588ab5a6-40ec-fa6b-9119-7cf13de98932, deleteOnExit=true 2023-02-09 10:00:22,900 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-09 10:00:22,900 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/test.cache.data in system properties and HBase conf 2023-02-09 10:00:22,900 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/hadoop.tmp.dir in system properties and HBase conf 2023-02-09 10:00:22,900 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/hadoop.log.dir in system properties and HBase conf 2023-02-09 10:00:22,901 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-09 10:00:22,901 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-09 10:00:22,901 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-09 10:00:22,901 DEBUG [Listener at localhost/33549] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-09 10:00:22,901 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-09 10:00:22,901 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-09 10:00:22,902 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-09 10:00:22,902 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 10:00:22,902 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-09 10:00:22,902 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-09 10:00:22,902 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 10:00:22,902 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 10:00:22,902 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-09 10:00:22,902 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/nfs.dump.dir in system properties and HBase conf 2023-02-09 10:00:22,903 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/java.io.tmpdir in system properties and HBase conf 2023-02-09 10:00:22,903 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 10:00:22,903 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-09 10:00:22,903 INFO [Listener at localhost/33549] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-09 10:00:22,907 WARN [Listener at localhost/33549] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 10:00:22,907 WARN [Listener at localhost/33549] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 10:00:22,956 WARN [Listener at localhost/33549] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 10:00:22,958 INFO [Listener at localhost/33549] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 10:00:22,962 INFO [Listener at localhost/33549] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/java.io.tmpdir/Jetty_localhost_35581_hdfs____.2c9mh6/webapp 2023-02-09 10:00:23,054 INFO [Listener at localhost/33549] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35581 2023-02-09 10:00:23,059 WARN [Listener at localhost/33549] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 10:00:23,059 WARN [Listener at localhost/33549] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 10:00:23,101 WARN [Listener at localhost/37725] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 10:00:23,112 WARN [Listener at localhost/37725] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 10:00:23,114 WARN [Listener at localhost/37725] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 10:00:23,115 INFO [Listener at localhost/37725] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 10:00:23,121 INFO [Listener at localhost/37725] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/java.io.tmpdir/Jetty_localhost_43331_datanode____wo5brt/webapp 2023-02-09 10:00:23,212 INFO [Listener at localhost/37725] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43331 2023-02-09 10:00:23,219 WARN [Listener at localhost/36099] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 10:00:23,235 WARN [Listener at localhost/36099] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 10:00:23,237 WARN [Listener at localhost/36099] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 10:00:23,238 INFO [Listener at localhost/36099] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 10:00:23,241 INFO [Listener at localhost/36099] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/java.io.tmpdir/Jetty_localhost_40339_datanode____j6jq6c/webapp 2023-02-09 10:00:23,325 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xbab2efdfdb4ce78: Processing first storage report for DS-8ca6abdc-1891-4a81-a87c-93f152698d0a from datanode 9b2c3394-e51a-4ac6-b18d-5e9681c7044f 2023-02-09 10:00:23,326 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xbab2efdfdb4ce78: from storage DS-8ca6abdc-1891-4a81-a87c-93f152698d0a node DatanodeRegistration(127.0.0.1:43463, datanodeUuid=9b2c3394-e51a-4ac6-b18d-5e9681c7044f, infoPort=41723, infoSecurePort=0, ipcPort=36099, storageInfo=lv=-57;cid=testClusterID;nsid=1189476638;c=1675936822910), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-09 10:00:23,326 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xbab2efdfdb4ce78: Processing first storage report for DS-a292c3e4-3a45-441a-bf98-d602192786fe from datanode 9b2c3394-e51a-4ac6-b18d-5e9681c7044f 2023-02-09 10:00:23,326 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xbab2efdfdb4ce78: from storage DS-a292c3e4-3a45-441a-bf98-d602192786fe node DatanodeRegistration(127.0.0.1:43463, datanodeUuid=9b2c3394-e51a-4ac6-b18d-5e9681c7044f, infoPort=41723, infoSecurePort=0, ipcPort=36099, storageInfo=lv=-57;cid=testClusterID;nsid=1189476638;c=1675936822910), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 10:00:23,342 INFO [Listener at localhost/36099] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40339 2023-02-09 10:00:23,351 WARN [Listener at localhost/33585] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 10:00:23,370 WARN [Listener at localhost/33585] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 10:00:23,372 WARN [Listener at localhost/33585] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 10:00:23,373 INFO [Listener at localhost/33585] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 10:00:23,378 INFO [Listener at localhost/33585] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/java.io.tmpdir/Jetty_localhost_42467_datanode____bukf1a/webapp 2023-02-09 10:00:23,450 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe9624dc8a8fb0a57: Processing first storage report for DS-e5210e12-8e61-4401-ad89-3f55ea035213 from datanode 2875a852-b6e6-4859-9f5d-c8c929854fe4 2023-02-09 10:00:23,451 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe9624dc8a8fb0a57: from storage DS-e5210e12-8e61-4401-ad89-3f55ea035213 node DatanodeRegistration(127.0.0.1:37779, datanodeUuid=2875a852-b6e6-4859-9f5d-c8c929854fe4, infoPort=44195, infoSecurePort=0, ipcPort=33585, storageInfo=lv=-57;cid=testClusterID;nsid=1189476638;c=1675936822910), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-09 10:00:23,451 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe9624dc8a8fb0a57: Processing first storage report for DS-77ce3bec-7a1a-4bb6-a804-fe081fbb272d from datanode 2875a852-b6e6-4859-9f5d-c8c929854fe4 2023-02-09 10:00:23,451 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe9624dc8a8fb0a57: from storage DS-77ce3bec-7a1a-4bb6-a804-fe081fbb272d node DatanodeRegistration(127.0.0.1:37779, datanodeUuid=2875a852-b6e6-4859-9f5d-c8c929854fe4, infoPort=44195, infoSecurePort=0, ipcPort=33585, storageInfo=lv=-57;cid=testClusterID;nsid=1189476638;c=1675936822910), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 10:00:23,482 INFO [Listener at localhost/33585] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42467 2023-02-09 10:00:23,491 WARN [Listener at localhost/34395] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 10:00:23,602 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x694c03a2bc16538b: Processing first storage report for DS-8784a953-bab3-4869-a930-897e10b4fa67 from datanode e39f1817-24ca-4e85-b648-9a6123e8186d 2023-02-09 10:00:23,602 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x694c03a2bc16538b: from storage DS-8784a953-bab3-4869-a930-897e10b4fa67 node DatanodeRegistration(127.0.0.1:39351, datanodeUuid=e39f1817-24ca-4e85-b648-9a6123e8186d, infoPort=39769, infoSecurePort=0, ipcPort=34395, storageInfo=lv=-57;cid=testClusterID;nsid=1189476638;c=1675936822910), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 10:00:23,602 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x694c03a2bc16538b: Processing first storage report for DS-5b17c28e-08b2-4e9e-b1f8-f24f77745c02 from datanode e39f1817-24ca-4e85-b648-9a6123e8186d 2023-02-09 10:00:23,602 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x694c03a2bc16538b: from storage DS-5b17c28e-08b2-4e9e-b1f8-f24f77745c02 node DatanodeRegistration(127.0.0.1:39351, datanodeUuid=e39f1817-24ca-4e85-b648-9a6123e8186d, infoPort=39769, infoSecurePort=0, ipcPort=34395, storageInfo=lv=-57;cid=testClusterID;nsid=1189476638;c=1675936822910), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 10:00:23,607 DEBUG [Listener at localhost/34395] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab 2023-02-09 10:00:23,611 INFO [Listener at localhost/34395] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/cluster_588ab5a6-40ec-fa6b-9119-7cf13de98932/zookeeper_0, clientPort=58825, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/cluster_588ab5a6-40ec-fa6b-9119-7cf13de98932/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/cluster_588ab5a6-40ec-fa6b-9119-7cf13de98932/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-09 10:00:23,613 INFO [Listener at localhost/34395] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=58825 2023-02-09 10:00:23,613 INFO [Listener at localhost/34395] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:23,614 INFO [Listener at localhost/34395] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:23,638 INFO [Listener at localhost/34395] util.FSUtils(479): Created version file at hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9 with version=8 2023-02-09 10:00:23,638 INFO [Listener at localhost/34395] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:34347/user/jenkins/test-data/e18dc7cd-306b-68f3-5c87-3c5279228cd6/hbase-staging 2023-02-09 10:00:23,640 INFO [Listener at localhost/34395] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=6 2023-02-09 10:00:23,641 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:23,641 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:23,641 INFO [Listener at localhost/34395] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 10:00:23,641 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:23,641 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 10:00:23,642 INFO [Listener at localhost/34395] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 10:00:23,644 INFO [Listener at localhost/34395] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33909 2023-02-09 10:00:23,645 INFO [Listener at localhost/34395] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:23,646 INFO [Listener at localhost/34395] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:23,646 INFO [Listener at localhost/34395] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33909 connecting to ZooKeeper ensemble=127.0.0.1:58825 2023-02-09 10:00:23,654 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:339090x0, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 10:00:23,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:33909-0x100388662ff0000 connected 2023-02-09 10:00:23,673 DEBUG [Listener at localhost/34395] zookeeper.ZKUtil(164): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 10:00:23,674 DEBUG [Listener at localhost/34395] zookeeper.ZKUtil(164): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 10:00:23,674 DEBUG [Listener at localhost/34395] zookeeper.ZKUtil(164): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 10:00:23,675 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33909 2023-02-09 10:00:23,676 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33909 2023-02-09 10:00:23,676 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33909 2023-02-09 10:00:23,678 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33909 2023-02-09 10:00:23,678 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33909 2023-02-09 10:00:23,679 INFO [Listener at localhost/34395] master.HMaster(439): hbase.rootdir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9, hbase.cluster.distributed=false 2023-02-09 10:00:23,692 INFO [Listener at localhost/34395] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-09 10:00:23,693 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:23,693 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:23,693 INFO [Listener at localhost/34395] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 10:00:23,693 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:23,693 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 10:00:23,693 INFO [Listener at localhost/34395] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 10:00:23,694 INFO [Listener at localhost/34395] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43217 2023-02-09 10:00:23,695 INFO [Listener at localhost/34395] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 10:00:23,696 DEBUG [Listener at localhost/34395] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 10:00:23,697 INFO [Listener at localhost/34395] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:23,698 INFO [Listener at localhost/34395] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:23,699 INFO [Listener at localhost/34395] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43217 connecting to ZooKeeper ensemble=127.0.0.1:58825 2023-02-09 10:00:23,703 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:432170x0, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 10:00:23,703 DEBUG [Listener at localhost/34395] zookeeper.ZKUtil(164): regionserver:432170x0, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 10:00:23,704 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:43217-0x100388662ff0001 connected 2023-02-09 10:00:23,704 DEBUG [Listener at localhost/34395] zookeeper.ZKUtil(164): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 10:00:23,705 DEBUG [Listener at localhost/34395] zookeeper.ZKUtil(164): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 10:00:23,705 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43217 2023-02-09 10:00:23,705 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43217 2023-02-09 10:00:23,706 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43217 2023-02-09 10:00:23,706 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43217 2023-02-09 10:00:23,706 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43217 2023-02-09 10:00:23,718 INFO [Listener at localhost/34395] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-09 10:00:23,718 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:23,718 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:23,718 INFO [Listener at localhost/34395] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 10:00:23,718 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:23,718 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 10:00:23,718 INFO [Listener at localhost/34395] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 10:00:23,720 INFO [Listener at localhost/34395] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35049 2023-02-09 10:00:23,720 INFO [Listener at localhost/34395] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 10:00:23,722 DEBUG [Listener at localhost/34395] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 10:00:23,723 INFO [Listener at localhost/34395] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:23,724 INFO [Listener at localhost/34395] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:23,725 INFO [Listener at localhost/34395] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35049 connecting to ZooKeeper ensemble=127.0.0.1:58825 2023-02-09 10:00:23,728 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:350490x0, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 10:00:23,729 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:35049-0x100388662ff0002 connected 2023-02-09 10:00:23,729 DEBUG [Listener at localhost/34395] zookeeper.ZKUtil(164): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 10:00:23,730 DEBUG [Listener at localhost/34395] zookeeper.ZKUtil(164): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 10:00:23,730 DEBUG [Listener at localhost/34395] zookeeper.ZKUtil(164): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 10:00:23,731 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35049 2023-02-09 10:00:23,732 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35049 2023-02-09 10:00:23,736 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35049 2023-02-09 10:00:23,736 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35049 2023-02-09 10:00:23,736 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35049 2023-02-09 10:00:23,747 INFO [Listener at localhost/34395] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-09 10:00:23,748 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:23,748 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:23,748 INFO [Listener at localhost/34395] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 10:00:23,748 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 10:00:23,748 INFO [Listener at localhost/34395] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 10:00:23,748 INFO [Listener at localhost/34395] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 10:00:23,750 INFO [Listener at localhost/34395] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38697 2023-02-09 10:00:23,750 INFO [Listener at localhost/34395] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 10:00:23,751 DEBUG [Listener at localhost/34395] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 10:00:23,752 INFO [Listener at localhost/34395] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:23,753 INFO [Listener at localhost/34395] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:23,754 INFO [Listener at localhost/34395] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38697 connecting to ZooKeeper ensemble=127.0.0.1:58825 2023-02-09 10:00:23,758 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:386970x0, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 10:00:23,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:38697-0x100388662ff0003 connected 2023-02-09 10:00:23,760 DEBUG [Listener at localhost/34395] zookeeper.ZKUtil(164): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 10:00:23,761 DEBUG [Listener at localhost/34395] zookeeper.ZKUtil(164): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 10:00:23,761 DEBUG [Listener at localhost/34395] zookeeper.ZKUtil(164): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 10:00:23,764 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38697 2023-02-09 10:00:23,764 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38697 2023-02-09 10:00:23,764 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38697 2023-02-09 10:00:23,767 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38697 2023-02-09 10:00:23,767 DEBUG [Listener at localhost/34395] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38697 2023-02-09 10:00:23,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,33909,1675936823640 2023-02-09 10:00:23,770 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 10:00:23,771 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,33909,1675936823640 2023-02-09 10:00:23,773 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 10:00:23,773 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 10:00:23,773 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 10:00:23,773 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 10:00:23,773 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:23,775 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 10:00:23,776 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,33909,1675936823640 from backup master directory 2023-02-09 10:00:23,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 10:00:23,778 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,33909,1675936823640 2023-02-09 10:00:23,778 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 10:00:23,778 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 10:00:23,778 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,33909,1675936823640 2023-02-09 10:00:23,802 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/hbase.id with ID: 157c8417-f004-4ef8-ab8d-9efc5d604c84 2023-02-09 10:00:23,817 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 10:00:23,821 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:23,836 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x09bc3a1b to 127.0.0.1:58825 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 10:00:23,840 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@520ce20f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 10:00:23,841 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-09 10:00:23,841 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-09 10:00:23,842 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 10:00:23,844 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/MasterData/data/master/store-tmp 2023-02-09 10:00:23,857 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 10:00:23,857 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 10:00:23,857 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:23,857 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:23,857 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 10:00:23,857 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:23,857 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:23,857 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675936823857Disabling compacts and flushes for region at 1675936823857Disabling writes for close at 1675936823857Writing region close event to WAL at 1675936823857Closed at 1675936823857 2023-02-09 10:00:23,858 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/MasterData/WALs/jenkins-hbase4.apache.org,33909,1675936823640 2023-02-09 10:00:23,863 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33909%2C1675936823640, suffix=, logDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/MasterData/WALs/jenkins-hbase4.apache.org,33909,1675936823640, archiveDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/MasterData/oldWALs, maxLogs=10 2023-02-09 10:00:23,879 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37779,DS-e5210e12-8e61-4401-ad89-3f55ea035213,DISK] 2023-02-09 10:00:23,880 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43463,DS-8ca6abdc-1891-4a81-a87c-93f152698d0a,DISK] 2023-02-09 10:00:23,880 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39351,DS-8784a953-bab3-4869-a930-897e10b4fa67,DISK] 2023-02-09 10:00:23,883 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/MasterData/WALs/jenkins-hbase4.apache.org,33909,1675936823640/jenkins-hbase4.apache.org%2C33909%2C1675936823640.1675936823863 2023-02-09 10:00:23,883 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37779,DS-e5210e12-8e61-4401-ad89-3f55ea035213,DISK], DatanodeInfoWithStorage[127.0.0.1:39351,DS-8784a953-bab3-4869-a930-897e10b4fa67,DISK], DatanodeInfoWithStorage[127.0.0.1:43463,DS-8ca6abdc-1891-4a81-a87c-93f152698d0a,DISK]] 2023-02-09 10:00:23,884 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-09 10:00:23,884 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 10:00:23,884 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 10:00:23,884 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 10:00:23,886 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-09 10:00:23,888 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-09 10:00:23,889 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-09 10:00:23,890 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:23,890 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 10:00:23,891 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 10:00:23,895 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 10:00:23,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 10:00:23,898 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70168826, jitterRate=0.04559698700904846}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 10:00:23,898 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675936823884Initializing all the Stores at 1675936823885 (+1 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675936823885Cleaning up temporary data from old regions at 1675936823892 (+7 ms)Cleaning up detritus from prior splits at 1675936823892Region opened successfully at 1675936823898 (+6 ms) 2023-02-09 10:00:23,898 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-09 10:00:23,899 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-09 10:00:23,899 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-09 10:00:23,899 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 10:00:23,900 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 10:00:23,900 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-02-09 10:00:23,900 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-09 10:00:23,901 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-09 10:00:23,902 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-09 10:00:23,913 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-09 10:00:23,913 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-09 10:00:23,914 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-09 10:00:23,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-09 10:00:23,914 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-09 10:00:23,917 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:23,918 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-09 10:00:23,918 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-09 10:00:23,919 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-09 10:00:23,920 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 10:00:23,920 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 10:00:23,920 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:23,920 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 10:00:23,920 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 10:00:23,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,33909,1675936823640, sessionid=0x100388662ff0000, setting cluster-up flag (Was=false) 2023-02-09 10:00:23,926 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:23,931 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-09 10:00:23,932 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33909,1675936823640 2023-02-09 10:00:23,936 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:23,940 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-09 10:00:23,941 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33909,1675936823640 2023-02-09 10:00:23,942 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/.hbase-snapshot/.tmp 2023-02-09 10:00:23,944 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-09 10:00:23,945 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 10:00:23,945 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 10:00:23,945 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 10:00:23,945 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 10:00:23,945 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-09 10:00:23,945 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:23,945 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 10:00:23,945 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:23,951 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675936853951 2023-02-09 10:00:23,951 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-09 10:00:23,951 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-09 10:00:23,951 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-09 10:00:23,951 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-09 10:00:23,952 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-09 10:00:23,952 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-09 10:00:23,952 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:23,952 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-09 10:00:23,953 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-09 10:00:23,953 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-09 10:00:23,953 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 10:00:23,953 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-09 10:00:23,953 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-09 10:00:23,953 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-09 10:00:23,953 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675936823953,5,FailOnTimeoutGroup] 2023-02-09 10:00:23,953 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675936823953,5,FailOnTimeoutGroup] 2023-02-09 10:00:23,954 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:23,954 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-09 10:00:23,954 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:23,954 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:23,955 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 10:00:23,970 INFO [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(952): ClusterId : 157c8417-f004-4ef8-ab8d-9efc5d604c84 2023-02-09 10:00:23,971 DEBUG [RS:0;jenkins-hbase4:43217] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 10:00:23,971 INFO [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(952): ClusterId : 157c8417-f004-4ef8-ab8d-9efc5d604c84 2023-02-09 10:00:23,971 INFO [RS:2;jenkins-hbase4:38697] regionserver.HRegionServer(952): ClusterId : 157c8417-f004-4ef8-ab8d-9efc5d604c84 2023-02-09 10:00:23,972 DEBUG [RS:1;jenkins-hbase4:35049] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 10:00:23,974 DEBUG [RS:2;jenkins-hbase4:38697] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 10:00:23,976 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 10:00:23,977 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 10:00:23,977 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9 2023-02-09 10:00:23,978 DEBUG [RS:0;jenkins-hbase4:43217] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 10:00:23,978 DEBUG [RS:1;jenkins-hbase4:35049] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 10:00:23,978 DEBUG [RS:1;jenkins-hbase4:35049] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 10:00:23,978 DEBUG [RS:0;jenkins-hbase4:43217] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 10:00:23,978 DEBUG [RS:2;jenkins-hbase4:38697] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 10:00:23,979 DEBUG [RS:2;jenkins-hbase4:38697] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 10:00:23,982 DEBUG [RS:1;jenkins-hbase4:35049] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 10:00:23,983 DEBUG [RS:2;jenkins-hbase4:38697] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 10:00:23,983 DEBUG [RS:0;jenkins-hbase4:43217] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 10:00:23,984 DEBUG [RS:1;jenkins-hbase4:35049] zookeeper.ReadOnlyZKClient(139): Connect 0x3427e4fe to 127.0.0.1:58825 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 10:00:23,984 DEBUG [RS:0;jenkins-hbase4:43217] zookeeper.ReadOnlyZKClient(139): Connect 0x35b38c4f to 127.0.0.1:58825 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 10:00:23,985 DEBUG [RS:2;jenkins-hbase4:38697] zookeeper.ReadOnlyZKClient(139): Connect 0x09622e5c to 127.0.0.1:58825 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 10:00:23,993 DEBUG [RS:1;jenkins-hbase4:35049] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@301866a2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 10:00:23,994 DEBUG [RS:2;jenkins-hbase4:38697] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@d5dbba8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 10:00:23,994 DEBUG [RS:0;jenkins-hbase4:43217] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3b747489, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 10:00:23,994 DEBUG [RS:1;jenkins-hbase4:35049] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@751aeec5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 10:00:23,994 DEBUG [RS:2;jenkins-hbase4:38697] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@61550ea4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 10:00:23,994 DEBUG [RS:0;jenkins-hbase4:43217] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7a497eb2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 10:00:23,998 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 10:00:24,003 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 10:00:24,004 DEBUG [RS:0;jenkins-hbase4:43217] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:43217 2023-02-09 10:00:24,004 INFO [RS:0;jenkins-hbase4:43217] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 10:00:24,004 INFO [RS:0;jenkins-hbase4:43217] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 10:00:24,004 DEBUG [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 10:00:24,005 DEBUG [RS:1;jenkins-hbase4:35049] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:35049 2023-02-09 10:00:24,005 DEBUG [RS:2;jenkins-hbase4:38697] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:38697 2023-02-09 10:00:24,005 INFO [RS:1;jenkins-hbase4:35049] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 10:00:24,005 INFO [RS:1;jenkins-hbase4:35049] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 10:00:24,005 INFO [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,33909,1675936823640 with isa=jenkins-hbase4.apache.org/172.31.14.131:43217, startcode=1675936823692 2023-02-09 10:00:24,005 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/info 2023-02-09 10:00:24,005 DEBUG [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 10:00:24,005 INFO [RS:2;jenkins-hbase4:38697] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 10:00:24,005 INFO [RS:2;jenkins-hbase4:38697] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 10:00:24,006 DEBUG [RS:2;jenkins-hbase4:38697] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 10:00:24,005 DEBUG [RS:0;jenkins-hbase4:43217] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 10:00:24,006 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 10:00:24,006 INFO [RS:2;jenkins-hbase4:38697] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,33909,1675936823640 with isa=jenkins-hbase4.apache.org/172.31.14.131:38697, startcode=1675936823747 2023-02-09 10:00:24,006 DEBUG [RS:2;jenkins-hbase4:38697] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 10:00:24,007 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:24,008 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 10:00:24,010 INFO [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,33909,1675936823640 with isa=jenkins-hbase4.apache.org/172.31.14.131:35049, startcode=1675936823717 2023-02-09 10:00:24,010 DEBUG [RS:1;jenkins-hbase4:35049] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 10:00:24,011 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60773, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 10:00:24,011 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/rep_barrier 2023-02-09 10:00:24,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33909] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:24,013 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35997, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 10:00:24,014 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 10:00:24,014 DEBUG [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9 2023-02-09 10:00:24,014 DEBUG [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:37725 2023-02-09 10:00:24,014 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33909] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:24,014 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52837, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 10:00:24,014 DEBUG [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 10:00:24,015 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33909] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38697,1675936823747 2023-02-09 10:00:24,015 DEBUG [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9 2023-02-09 10:00:24,015 DEBUG [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:37725 2023-02-09 10:00:24,015 DEBUG [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 10:00:24,016 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:24,016 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 10:00:24,016 DEBUG [RS:2;jenkins-hbase4:38697] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9 2023-02-09 10:00:24,016 DEBUG [RS:2;jenkins-hbase4:38697] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:37725 2023-02-09 10:00:24,017 DEBUG [RS:2;jenkins-hbase4:38697] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 10:00:24,020 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/table 2023-02-09 10:00:24,020 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:24,021 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 10:00:24,022 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:24,023 DEBUG [RS:0;jenkins-hbase4:43217] zookeeper.ZKUtil(162): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:24,023 DEBUG [RS:2;jenkins-hbase4:38697] zookeeper.ZKUtil(162): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38697,1675936823747 2023-02-09 10:00:24,023 DEBUG [RS:1;jenkins-hbase4:35049] zookeeper.ZKUtil(162): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:24,025 WARN [RS:2;jenkins-hbase4:38697] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 10:00:24,024 WARN [RS:0;jenkins-hbase4:43217] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 10:00:24,025 INFO [RS:2;jenkins-hbase4:38697] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 10:00:24,025 WARN [RS:1;jenkins-hbase4:35049] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 10:00:24,025 INFO [RS:0;jenkins-hbase4:43217] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 10:00:24,025 INFO [RS:1;jenkins-hbase4:35049] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 10:00:24,025 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38697,1675936823747] 2023-02-09 10:00:24,026 DEBUG [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(1947): logDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/WALs/jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:24,026 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740 2023-02-09 10:00:24,025 DEBUG [RS:2;jenkins-hbase4:38697] regionserver.HRegionServer(1947): logDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/WALs/jenkins-hbase4.apache.org,38697,1675936823747 2023-02-09 10:00:24,026 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,35049,1675936823717] 2023-02-09 10:00:24,026 DEBUG [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(1947): logDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/WALs/jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:24,026 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43217,1675936823692] 2023-02-09 10:00:24,026 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740 2023-02-09 10:00:24,032 DEBUG [RS:0;jenkins-hbase4:43217] zookeeper.ZKUtil(162): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:24,033 DEBUG [RS:2;jenkins-hbase4:38697] zookeeper.ZKUtil(162): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:24,033 DEBUG [RS:1;jenkins-hbase4:35049] zookeeper.ZKUtil(162): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:24,033 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 10:00:24,033 DEBUG [RS:2;jenkins-hbase4:38697] zookeeper.ZKUtil(162): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38697,1675936823747 2023-02-09 10:00:24,033 DEBUG [RS:0;jenkins-hbase4:43217] zookeeper.ZKUtil(162): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38697,1675936823747 2023-02-09 10:00:24,034 DEBUG [RS:2;jenkins-hbase4:38697] zookeeper.ZKUtil(162): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:24,035 DEBUG [RS:0;jenkins-hbase4:43217] zookeeper.ZKUtil(162): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:24,035 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 10:00:24,035 DEBUG [RS:2;jenkins-hbase4:38697] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 10:00:24,036 DEBUG [RS:0;jenkins-hbase4:43217] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 10:00:24,036 INFO [RS:2;jenkins-hbase4:38697] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 10:00:24,036 INFO [RS:0;jenkins-hbase4:43217] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 10:00:24,038 INFO [RS:2;jenkins-hbase4:38697] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 10:00:24,039 INFO [RS:2;jenkins-hbase4:38697] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 10:00:24,039 INFO [RS:2;jenkins-hbase4:38697] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,039 INFO [RS:2;jenkins-hbase4:38697] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 10:00:24,040 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 10:00:24,041 DEBUG [RS:1;jenkins-hbase4:35049] zookeeper.ZKUtil(162): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38697,1675936823747 2023-02-09 10:00:24,041 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64266424, jitterRate=-0.04235565662384033}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 10:00:24,041 INFO [RS:2;jenkins-hbase4:38697] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,041 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675936823998Initializing all the Stores at 1675936823999 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936823999Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675936824003 (+4 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936824003Cleaning up temporary data from old regions at 1675936824031 (+28 ms)Cleaning up detritus from prior splits at 1675936824032 (+1 ms)Region opened successfully at 1675936824041 (+9 ms) 2023-02-09 10:00:24,041 DEBUG [RS:2;jenkins-hbase4:38697] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,041 DEBUG [RS:1;jenkins-hbase4:35049] zookeeper.ZKUtil(162): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:24,042 DEBUG [RS:2;jenkins-hbase4:38697] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,042 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 10:00:24,042 DEBUG [RS:2;jenkins-hbase4:38697] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,042 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 10:00:24,043 DEBUG [RS:2;jenkins-hbase4:38697] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,043 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 10:00:24,043 DEBUG [RS:2;jenkins-hbase4:38697] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,043 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 10:00:24,043 DEBUG [RS:2;jenkins-hbase4:38697] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 10:00:24,043 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 10:00:24,043 DEBUG [RS:2;jenkins-hbase4:38697] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,043 DEBUG [RS:2;jenkins-hbase4:38697] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,043 DEBUG [RS:1;jenkins-hbase4:35049] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 10:00:24,043 DEBUG [RS:2;jenkins-hbase4:38697] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,043 DEBUG [RS:2;jenkins-hbase4:38697] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,043 INFO [RS:1;jenkins-hbase4:35049] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 10:00:24,051 INFO [RS:0;jenkins-hbase4:43217] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 10:00:24,052 INFO [RS:1;jenkins-hbase4:35049] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 10:00:24,053 INFO [RS:0;jenkins-hbase4:43217] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 10:00:24,053 INFO [RS:0;jenkins-hbase4:43217] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,053 INFO [RS:1;jenkins-hbase4:35049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 10:00:24,053 INFO [RS:1;jenkins-hbase4:35049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,054 INFO [RS:2;jenkins-hbase4:38697] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,054 INFO [RS:2;jenkins-hbase4:38697] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,054 INFO [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 10:00:24,054 INFO [RS:2;jenkins-hbase4:38697] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,055 INFO [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 10:00:24,055 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 10:00:24,056 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675936824042Disabling compacts and flushes for region at 1675936824042Disabling writes for close at 1675936824043 (+1 ms)Writing region close event to WAL at 1675936824055 (+12 ms)Closed at 1675936824055 2023-02-09 10:00:24,057 INFO [RS:0;jenkins-hbase4:43217] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,057 INFO [RS:1;jenkins-hbase4:35049] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,057 DEBUG [RS:0;jenkins-hbase4:43217] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,057 DEBUG [RS:1;jenkins-hbase4:35049] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,057 DEBUG [RS:0;jenkins-hbase4:43217] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,057 DEBUG [RS:1;jenkins-hbase4:35049] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,057 DEBUG [RS:0;jenkins-hbase4:43217] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,057 DEBUG [RS:1;jenkins-hbase4:35049] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,057 DEBUG [RS:0;jenkins-hbase4:43217] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,057 DEBUG [RS:1;jenkins-hbase4:35049] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,057 DEBUG [RS:0;jenkins-hbase4:43217] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,057 DEBUG [RS:1;jenkins-hbase4:35049] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,058 DEBUG [RS:0;jenkins-hbase4:43217] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 10:00:24,058 DEBUG [RS:1;jenkins-hbase4:35049] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 10:00:24,058 DEBUG [RS:0;jenkins-hbase4:43217] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,058 DEBUG [RS:1;jenkins-hbase4:35049] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,058 DEBUG [RS:0;jenkins-hbase4:43217] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,058 DEBUG [RS:1;jenkins-hbase4:35049] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,058 DEBUG [RS:0;jenkins-hbase4:43217] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,058 DEBUG [RS:1;jenkins-hbase4:35049] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,058 DEBUG [RS:0;jenkins-hbase4:43217] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,058 DEBUG [RS:1;jenkins-hbase4:35049] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 10:00:24,059 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 10:00:24,059 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-09 10:00:24,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-09 10:00:24,070 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-09 10:00:24,072 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-09 10:00:24,072 INFO [RS:0;jenkins-hbase4:43217] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,072 INFO [RS:0;jenkins-hbase4:43217] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,072 INFO [RS:0;jenkins-hbase4:43217] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,074 INFO [RS:2;jenkins-hbase4:38697] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 10:00:24,075 INFO [RS:2;jenkins-hbase4:38697] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38697,1675936823747-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,076 INFO [RS:1;jenkins-hbase4:35049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,076 INFO [RS:1;jenkins-hbase4:35049] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,076 INFO [RS:1;jenkins-hbase4:35049] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,088 INFO [RS:0;jenkins-hbase4:43217] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 10:00:24,088 INFO [RS:0;jenkins-hbase4:43217] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43217,1675936823692-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,089 INFO [RS:2;jenkins-hbase4:38697] regionserver.Replication(203): jenkins-hbase4.apache.org,38697,1675936823747 started 2023-02-09 10:00:24,089 INFO [RS:2;jenkins-hbase4:38697] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,38697,1675936823747, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38697, sessionid=0x100388662ff0003 2023-02-09 10:00:24,089 DEBUG [RS:2;jenkins-hbase4:38697] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 10:00:24,089 DEBUG [RS:2;jenkins-hbase4:38697] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38697,1675936823747 2023-02-09 10:00:24,089 DEBUG [RS:2;jenkins-hbase4:38697] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38697,1675936823747' 2023-02-09 10:00:24,089 DEBUG [RS:2;jenkins-hbase4:38697] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 10:00:24,090 DEBUG [RS:2;jenkins-hbase4:38697] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 10:00:24,090 DEBUG [RS:2;jenkins-hbase4:38697] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 10:00:24,090 DEBUG [RS:2;jenkins-hbase4:38697] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 10:00:24,090 DEBUG [RS:2;jenkins-hbase4:38697] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38697,1675936823747 2023-02-09 10:00:24,090 DEBUG [RS:2;jenkins-hbase4:38697] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38697,1675936823747' 2023-02-09 10:00:24,090 DEBUG [RS:2;jenkins-hbase4:38697] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 10:00:24,091 DEBUG [RS:2;jenkins-hbase4:38697] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 10:00:24,091 DEBUG [RS:2;jenkins-hbase4:38697] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 10:00:24,091 INFO [RS:2;jenkins-hbase4:38697] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 10:00:24,091 INFO [RS:2;jenkins-hbase4:38697] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 10:00:24,096 INFO [RS:1;jenkins-hbase4:35049] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 10:00:24,097 INFO [RS:1;jenkins-hbase4:35049] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35049,1675936823717-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,104 INFO [RS:0;jenkins-hbase4:43217] regionserver.Replication(203): jenkins-hbase4.apache.org,43217,1675936823692 started 2023-02-09 10:00:24,104 INFO [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,43217,1675936823692, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43217, sessionid=0x100388662ff0001 2023-02-09 10:00:24,104 DEBUG [RS:0;jenkins-hbase4:43217] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 10:00:24,104 DEBUG [RS:0;jenkins-hbase4:43217] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:24,104 DEBUG [RS:0;jenkins-hbase4:43217] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43217,1675936823692' 2023-02-09 10:00:24,104 DEBUG [RS:0;jenkins-hbase4:43217] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 10:00:24,105 DEBUG [RS:0;jenkins-hbase4:43217] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 10:00:24,105 DEBUG [RS:0;jenkins-hbase4:43217] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 10:00:24,105 DEBUG [RS:0;jenkins-hbase4:43217] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 10:00:24,105 DEBUG [RS:0;jenkins-hbase4:43217] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:24,105 DEBUG [RS:0;jenkins-hbase4:43217] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43217,1675936823692' 2023-02-09 10:00:24,105 DEBUG [RS:0;jenkins-hbase4:43217] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 10:00:24,105 DEBUG [RS:0;jenkins-hbase4:43217] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 10:00:24,106 DEBUG [RS:0;jenkins-hbase4:43217] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 10:00:24,106 INFO [RS:0;jenkins-hbase4:43217] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 10:00:24,106 INFO [RS:0;jenkins-hbase4:43217] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 10:00:24,116 INFO [RS:1;jenkins-hbase4:35049] regionserver.Replication(203): jenkins-hbase4.apache.org,35049,1675936823717 started 2023-02-09 10:00:24,116 INFO [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,35049,1675936823717, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:35049, sessionid=0x100388662ff0002 2023-02-09 10:00:24,116 DEBUG [RS:1;jenkins-hbase4:35049] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 10:00:24,116 DEBUG [RS:1;jenkins-hbase4:35049] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:24,116 DEBUG [RS:1;jenkins-hbase4:35049] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35049,1675936823717' 2023-02-09 10:00:24,116 DEBUG [RS:1;jenkins-hbase4:35049] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 10:00:24,116 DEBUG [RS:1;jenkins-hbase4:35049] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 10:00:24,117 DEBUG [RS:1;jenkins-hbase4:35049] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 10:00:24,117 DEBUG [RS:1;jenkins-hbase4:35049] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 10:00:24,117 DEBUG [RS:1;jenkins-hbase4:35049] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:24,117 DEBUG [RS:1;jenkins-hbase4:35049] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35049,1675936823717' 2023-02-09 10:00:24,117 DEBUG [RS:1;jenkins-hbase4:35049] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 10:00:24,117 DEBUG [RS:1;jenkins-hbase4:35049] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 10:00:24,118 DEBUG [RS:1;jenkins-hbase4:35049] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 10:00:24,118 INFO [RS:1;jenkins-hbase4:35049] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 10:00:24,118 INFO [RS:1;jenkins-hbase4:35049] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 10:00:24,194 INFO [RS:2;jenkins-hbase4:38697] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38697%2C1675936823747, suffix=, logDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/WALs/jenkins-hbase4.apache.org,38697,1675936823747, archiveDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/oldWALs, maxLogs=32 2023-02-09 10:00:24,211 INFO [RS:0;jenkins-hbase4:43217] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43217%2C1675936823692, suffix=, logDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/WALs/jenkins-hbase4.apache.org,43217,1675936823692, archiveDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/oldWALs, maxLogs=32 2023-02-09 10:00:24,216 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37779,DS-e5210e12-8e61-4401-ad89-3f55ea035213,DISK] 2023-02-09 10:00:24,216 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43463,DS-8ca6abdc-1891-4a81-a87c-93f152698d0a,DISK] 2023-02-09 10:00:24,216 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39351,DS-8784a953-bab3-4869-a930-897e10b4fa67,DISK] 2023-02-09 10:00:24,222 INFO [RS:2;jenkins-hbase4:38697] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/WALs/jenkins-hbase4.apache.org,38697,1675936823747/jenkins-hbase4.apache.org%2C38697%2C1675936823747.1675936824195 2023-02-09 10:00:24,222 DEBUG [RS:2;jenkins-hbase4:38697] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37779,DS-e5210e12-8e61-4401-ad89-3f55ea035213,DISK], DatanodeInfoWithStorage[127.0.0.1:43463,DS-8ca6abdc-1891-4a81-a87c-93f152698d0a,DISK], DatanodeInfoWithStorage[127.0.0.1:39351,DS-8784a953-bab3-4869-a930-897e10b4fa67,DISK]] 2023-02-09 10:00:24,224 INFO [RS:1;jenkins-hbase4:35049] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35049%2C1675936823717, suffix=, logDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/WALs/jenkins-hbase4.apache.org,35049,1675936823717, archiveDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/oldWALs, maxLogs=32 2023-02-09 10:00:24,226 DEBUG [jenkins-hbase4:33909] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-09 10:00:24,227 DEBUG [jenkins-hbase4:33909] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 10:00:24,229 DEBUG [jenkins-hbase4:33909] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 10:00:24,229 DEBUG [jenkins-hbase4:33909] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 10:00:24,229 DEBUG [jenkins-hbase4:33909] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 10:00:24,229 DEBUG [jenkins-hbase4:33909] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 10:00:24,231 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,35049,1675936823717, state=OPENING 2023-02-09 10:00:24,237 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-09 10:00:24,237 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37779,DS-e5210e12-8e61-4401-ad89-3f55ea035213,DISK] 2023-02-09 10:00:24,237 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39351,DS-8784a953-bab3-4869-a930-897e10b4fa67,DISK] 2023-02-09 10:00:24,238 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43463,DS-8ca6abdc-1891-4a81-a87c-93f152698d0a,DISK] 2023-02-09 10:00:24,239 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:24,239 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,35049,1675936823717}] 2023-02-09 10:00:24,239 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 10:00:24,249 INFO [RS:0;jenkins-hbase4:43217] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/WALs/jenkins-hbase4.apache.org,43217,1675936823692/jenkins-hbase4.apache.org%2C43217%2C1675936823692.1675936824214 2023-02-09 10:00:24,249 DEBUG [RS:0;jenkins-hbase4:43217] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43463,DS-8ca6abdc-1891-4a81-a87c-93f152698d0a,DISK], DatanodeInfoWithStorage[127.0.0.1:39351,DS-8784a953-bab3-4869-a930-897e10b4fa67,DISK], DatanodeInfoWithStorage[127.0.0.1:37779,DS-e5210e12-8e61-4401-ad89-3f55ea035213,DISK]] 2023-02-09 10:00:24,254 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39351,DS-8784a953-bab3-4869-a930-897e10b4fa67,DISK] 2023-02-09 10:00:24,254 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43463,DS-8ca6abdc-1891-4a81-a87c-93f152698d0a,DISK] 2023-02-09 10:00:24,254 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37779,DS-e5210e12-8e61-4401-ad89-3f55ea035213,DISK] 2023-02-09 10:00:24,260 INFO [RS:1;jenkins-hbase4:35049] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/WALs/jenkins-hbase4.apache.org,35049,1675936823717/jenkins-hbase4.apache.org%2C35049%2C1675936823717.1675936824225 2023-02-09 10:00:24,260 DEBUG [RS:1;jenkins-hbase4:35049] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43463,DS-8ca6abdc-1891-4a81-a87c-93f152698d0a,DISK], DatanodeInfoWithStorage[127.0.0.1:37779,DS-e5210e12-8e61-4401-ad89-3f55ea035213,DISK], DatanodeInfoWithStorage[127.0.0.1:39351,DS-8784a953-bab3-4869-a930-897e10b4fa67,DISK]] 2023-02-09 10:00:24,401 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:24,402 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 10:00:24,403 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51328, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 10:00:24,409 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-09 10:00:24,409 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 10:00:24,411 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35049%2C1675936823717.meta, suffix=.meta, logDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/WALs/jenkins-hbase4.apache.org,35049,1675936823717, archiveDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/oldWALs, maxLogs=32 2023-02-09 10:00:24,426 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43463,DS-8ca6abdc-1891-4a81-a87c-93f152698d0a,DISK] 2023-02-09 10:00:24,428 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37779,DS-e5210e12-8e61-4401-ad89-3f55ea035213,DISK] 2023-02-09 10:00:24,428 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39351,DS-8784a953-bab3-4869-a930-897e10b4fa67,DISK] 2023-02-09 10:00:24,431 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/WALs/jenkins-hbase4.apache.org,35049,1675936823717/jenkins-hbase4.apache.org%2C35049%2C1675936823717.meta.1675936824412.meta 2023-02-09 10:00:24,433 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43463,DS-8ca6abdc-1891-4a81-a87c-93f152698d0a,DISK], DatanodeInfoWithStorage[127.0.0.1:39351,DS-8784a953-bab3-4869-a930-897e10b4fa67,DISK], DatanodeInfoWithStorage[127.0.0.1:37779,DS-e5210e12-8e61-4401-ad89-3f55ea035213,DISK]] 2023-02-09 10:00:24,433 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-09 10:00:24,433 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-09 10:00:24,433 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-09 10:00:24,436 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-09 10:00:24,436 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-09 10:00:24,436 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 10:00:24,436 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-09 10:00:24,436 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-09 10:00:24,438 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 10:00:24,439 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/info 2023-02-09 10:00:24,439 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/info 2023-02-09 10:00:24,440 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 10:00:24,441 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:24,441 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 10:00:24,442 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/rep_barrier 2023-02-09 10:00:24,442 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/rep_barrier 2023-02-09 10:00:24,443 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 10:00:24,444 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:24,444 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 10:00:24,445 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/table 2023-02-09 10:00:24,445 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/table 2023-02-09 10:00:24,445 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 10:00:24,446 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:24,447 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740 2023-02-09 10:00:24,449 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740 2023-02-09 10:00:24,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 10:00:24,454 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 10:00:24,455 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59765390, jitterRate=-0.10942628979682922}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 10:00:24,455 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675936824436Writing region info on filesystem at 1675936824436Initializing all the Stores at 1675936824437 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936824438 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675936824438Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936824438Cleaning up temporary data from old regions at 1675936824450 (+12 ms)Cleaning up detritus from prior splits at 1675936824452 (+2 ms)Running coprocessor post-open hooks at 1675936824455 (+3 ms)Region opened successfully at 1675936824455 2023-02-09 10:00:24,457 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675936824401 2023-02-09 10:00:24,462 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-09 10:00:24,463 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-09 10:00:24,464 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,35049,1675936823717, state=OPEN 2023-02-09 10:00:24,467 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-09 10:00:24,467 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 10:00:24,470 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-09 10:00:24,470 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,35049,1675936823717 in 228 msec 2023-02-09 10:00:24,473 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-09 10:00:24,473 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 411 msec 2023-02-09 10:00:24,475 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 531 msec 2023-02-09 10:00:24,476 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675936823771, completionTime=-1 2023-02-09 10:00:24,476 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-09 10:00:24,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 10:00:24,478 DEBUG [hconnection-0x58c33d82-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 10:00:24,480 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51342, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 10:00:24,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 10:00:24,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675936884481 2023-02-09 10:00:24,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675936944481 2023-02-09 10:00:24,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-09 10:00:24,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33909,1675936823640-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33909,1675936823640-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,490 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33909,1675936823640-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,490 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33909, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,490 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-09 10:00:24,490 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-09 10:00:24,490 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 10:00:24,491 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-09 10:00:24,492 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-09 10:00:24,493 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 10:00:24,494 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 10:00:24,497 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/.tmp/data/hbase/namespace/958c765f00213055f1a0dc052434ad89 2023-02-09 10:00:24,497 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/.tmp/data/hbase/namespace/958c765f00213055f1a0dc052434ad89 empty. 2023-02-09 10:00:24,498 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/.tmp/data/hbase/namespace/958c765f00213055f1a0dc052434ad89 2023-02-09 10:00:24,498 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-09 10:00:24,514 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-09 10:00:24,516 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 958c765f00213055f1a0dc052434ad89, NAME => 'hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/.tmp 2023-02-09 10:00:24,531 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 10:00:24,531 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 958c765f00213055f1a0dc052434ad89, disabling compactions & flushes 2023-02-09 10:00:24,531 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. 2023-02-09 10:00:24,531 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. 2023-02-09 10:00:24,531 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. after waiting 0 ms 2023-02-09 10:00:24,531 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. 2023-02-09 10:00:24,531 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. 2023-02-09 10:00:24,532 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 958c765f00213055f1a0dc052434ad89: Waiting for close lock at 1675936824531Disabling compacts and flushes for region at 1675936824531Disabling writes for close at 1675936824531Writing region close event to WAL at 1675936824531Closed at 1675936824531 2023-02-09 10:00:24,535 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 10:00:24,536 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675936824536"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936824536"}]},"ts":"1675936824536"} 2023-02-09 10:00:24,539 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 10:00:24,540 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 10:00:24,541 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936824540"}]},"ts":"1675936824540"} 2023-02-09 10:00:24,542 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-09 10:00:24,546 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 10:00:24,547 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 10:00:24,547 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 10:00:24,547 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 10:00:24,547 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 10:00:24,547 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=958c765f00213055f1a0dc052434ad89, ASSIGN}] 2023-02-09 10:00:24,550 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=958c765f00213055f1a0dc052434ad89, ASSIGN 2023-02-09 10:00:24,551 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=958c765f00213055f1a0dc052434ad89, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43217,1675936823692; forceNewPlan=false, retain=false 2023-02-09 10:00:24,701 INFO [jenkins-hbase4:33909] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 10:00:24,702 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=958c765f00213055f1a0dc052434ad89, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:24,702 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675936824702"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936824702"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936824702"}]},"ts":"1675936824702"} 2023-02-09 10:00:24,705 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 958c765f00213055f1a0dc052434ad89, server=jenkins-hbase4.apache.org,43217,1675936823692}] 2023-02-09 10:00:24,859 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:24,860 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 10:00:24,861 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50894, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 10:00:24,866 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. 2023-02-09 10:00:24,866 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 958c765f00213055f1a0dc052434ad89, NAME => 'hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89.', STARTKEY => '', ENDKEY => ''} 2023-02-09 10:00:24,866 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 958c765f00213055f1a0dc052434ad89 2023-02-09 10:00:24,866 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 10:00:24,866 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 958c765f00213055f1a0dc052434ad89 2023-02-09 10:00:24,866 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 958c765f00213055f1a0dc052434ad89 2023-02-09 10:00:24,868 INFO [StoreOpener-958c765f00213055f1a0dc052434ad89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 958c765f00213055f1a0dc052434ad89 2023-02-09 10:00:24,870 DEBUG [StoreOpener-958c765f00213055f1a0dc052434ad89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/namespace/958c765f00213055f1a0dc052434ad89/info 2023-02-09 10:00:24,870 DEBUG [StoreOpener-958c765f00213055f1a0dc052434ad89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/namespace/958c765f00213055f1a0dc052434ad89/info 2023-02-09 10:00:24,871 INFO [StoreOpener-958c765f00213055f1a0dc052434ad89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 958c765f00213055f1a0dc052434ad89 columnFamilyName info 2023-02-09 10:00:24,872 INFO [StoreOpener-958c765f00213055f1a0dc052434ad89-1] regionserver.HStore(310): Store=958c765f00213055f1a0dc052434ad89/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 10:00:24,873 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/namespace/958c765f00213055f1a0dc052434ad89 2023-02-09 10:00:24,874 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/namespace/958c765f00213055f1a0dc052434ad89 2023-02-09 10:00:24,877 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 958c765f00213055f1a0dc052434ad89 2023-02-09 10:00:24,879 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/namespace/958c765f00213055f1a0dc052434ad89/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 10:00:24,880 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 958c765f00213055f1a0dc052434ad89; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59965492, jitterRate=-0.10644453763961792}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 10:00:24,880 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 958c765f00213055f1a0dc052434ad89: Running coprocessor pre-open hook at 1675936824866Writing region info on filesystem at 1675936824866Initializing all the Stores at 1675936824867 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936824867Cleaning up temporary data from old regions at 1675936824874 (+7 ms)Cleaning up detritus from prior splits at 1675936824875 (+1 ms)Running coprocessor post-open hooks at 1675936824880 (+5 ms)Region opened successfully at 1675936824880 2023-02-09 10:00:24,881 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89., pid=6, masterSystemTime=1675936824859 2023-02-09 10:00:24,885 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. 2023-02-09 10:00:24,885 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. 2023-02-09 10:00:24,886 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=958c765f00213055f1a0dc052434ad89, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:24,887 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675936824886"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936824886"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936824886"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936824886"}]},"ts":"1675936824886"} 2023-02-09 10:00:24,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-09 10:00:24,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 958c765f00213055f1a0dc052434ad89, server=jenkins-hbase4.apache.org,43217,1675936823692 in 184 msec 2023-02-09 10:00:24,896 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-09 10:00:24,897 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=958c765f00213055f1a0dc052434ad89, ASSIGN in 345 msec 2023-02-09 10:00:24,898 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 10:00:24,898 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936824898"}]},"ts":"1675936824898"} 2023-02-09 10:00:24,900 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-09 10:00:24,904 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 10:00:24,906 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 414 msec 2023-02-09 10:00:24,993 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-09 10:00:24,994 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-09 10:00:24,994 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:24,998 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 10:00:24,999 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50898, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 10:00:25,002 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-09 10:00:25,014 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 10:00:25,019 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 16 msec 2023-02-09 10:00:25,025 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-09 10:00:25,035 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 10:00:25,040 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 13 msec 2023-02-09 10:00:25,049 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-09 10:00:25,053 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-09 10:00:25,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 1.275sec 2023-02-09 10:00:25,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-09 10:00:25,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-09 10:00:25,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-09 10:00:25,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33909,1675936823640-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-09 10:00:25,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33909,1675936823640-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-09 10:00:25,055 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-09 10:00:25,070 DEBUG [Listener at localhost/34395] zookeeper.ReadOnlyZKClient(139): Connect 0x71d9a02f to 127.0.0.1:58825 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 10:00:25,075 DEBUG [Listener at localhost/34395] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5c7748f1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 10:00:25,077 DEBUG [hconnection-0x36ae13be-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 10:00:25,079 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51356, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 10:00:25,081 INFO [Listener at localhost/34395] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,33909,1675936823640 2023-02-09 10:00:25,082 DEBUG [Listener at localhost/34395] zookeeper.ReadOnlyZKClient(139): Connect 0x206f8695 to 127.0.0.1:58825 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 10:00:25,088 DEBUG [ReadOnlyZKClient-127.0.0.1:58825@0x206f8695] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@268b0713, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 10:00:25,089 DEBUG [Listener at localhost/34395] client.ConnectionUtils(586): Start fetching master stub from registry 2023-02-09 10:00:25,090 DEBUG [ReadOnlyZKClient-127.0.0.1:58825@0x206f8695] client.AsyncConnectionImpl(289): The fetched master address is jenkins-hbase4.apache.org,33909,1675936823640 2023-02-09 10:00:25,090 DEBUG [ReadOnlyZKClient-127.0.0.1:58825@0x206f8695] client.ConnectionUtils(594): The fetched master stub is org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$Stub@22b4871a 2023-02-09 10:00:25,093 DEBUG [ReadOnlyZKClient-127.0.0.1:58825@0x206f8695] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 10:00:25,095 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55970, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 10:00:25,095 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33909] master.MasterRpcServices(1560): Client=jenkins//172.31.14.131 shutdown 2023-02-09 10:00:25,095 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33909] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33909,1675936823640 2023-02-09 10:00:25,098 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 10:00:25,098 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 10:00:25,098 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 10:00:25,098 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 10:00:25,098 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33909] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 10:00:25,098 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:25,099 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 10:00:25,100 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33909] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x09bc3a1b to 127.0.0.1:58825 2023-02-09 10:00:25,100 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 10:00:25,100 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 10:00:25,100 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33909] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:25,100 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 10:00:25,137 INFO [RS:2;jenkins-hbase4:38697] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,38697,1675936823747' ***** 2023-02-09 10:00:25,138 INFO [RS:2;jenkins-hbase4:38697] regionserver.HRegionServer(2310): STOPPED: Exiting; cluster shutdown set and not carrying any regions 2023-02-09 10:00:25,139 INFO [RS:2;jenkins-hbase4:38697] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 10:00:25,139 INFO [RS:2;jenkins-hbase4:38697] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 10:00:25,139 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 10:00:25,139 INFO [RS:2;jenkins-hbase4:38697] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 10:00:25,140 INFO [RS:2;jenkins-hbase4:38697] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,38697,1675936823747 2023-02-09 10:00:25,140 DEBUG [RS:2;jenkins-hbase4:38697] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x09622e5c to 127.0.0.1:58825 2023-02-09 10:00:25,140 DEBUG [RS:2;jenkins-hbase4:38697] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:25,141 INFO [RS:2;jenkins-hbase4:38697] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,38697,1675936823747; all regions closed. 2023-02-09 10:00:25,148 DEBUG [RS:2;jenkins-hbase4:38697] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/oldWALs 2023-02-09 10:00:25,148 INFO [RS:2;jenkins-hbase4:38697] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38697%2C1675936823747:(num 1675936824195) 2023-02-09 10:00:25,148 DEBUG [RS:2;jenkins-hbase4:38697] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:25,148 INFO [RS:2;jenkins-hbase4:38697] regionserver.LeaseManager(133): Closed leases 2023-02-09 10:00:25,148 INFO [RS:2;jenkins-hbase4:38697] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-09 10:00:25,149 INFO [RS:2;jenkins-hbase4:38697] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 10:00:25,149 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 10:00:25,149 INFO [RS:2;jenkins-hbase4:38697] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 10:00:25,149 INFO [RS:2;jenkins-hbase4:38697] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 10:00:25,149 INFO [RS:2;jenkins-hbase4:38697] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38697 2023-02-09 10:00:25,153 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38697,1675936823747 2023-02-09 10:00:25,153 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38697,1675936823747 2023-02-09 10:00:25,153 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:25,153 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:25,153 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:25,154 ERROR [Listener at localhost/34395-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2d6454e3 rejected from java.util.concurrent.ThreadPoolExecutor@295bc597[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 5] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:25,153 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38697,1675936823747 2023-02-09 10:00:25,154 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:25,156 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38697,1675936823747] 2023-02-09 10:00:25,156 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:25,156 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:25,156 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38697,1675936823747; numProcessing=1 2023-02-09 10:00:25,156 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:25,158 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 10:00:25,158 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:25,159 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,38697,1675936823747 znode expired, triggering replicatorRemoved event 2023-02-09 10:00:25,159 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,38697,1675936823747 znode expired, triggering replicatorRemoved event 2023-02-09 10:00:25,159 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,38697,1675936823747 already deleted, retry=false 2023-02-09 10:00:25,159 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38697,1675936823747 expired; onlineServers=2 2023-02-09 10:00:25,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:25,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:25,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:25,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:25,163 INFO [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(1065): Closing user regions 2023-02-09 10:00:25,163 INFO [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(3304): Received CLOSE for 958c765f00213055f1a0dc052434ad89 2023-02-09 10:00:25,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 958c765f00213055f1a0dc052434ad89, disabling compactions & flushes 2023-02-09 10:00:25,164 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. 2023-02-09 10:00:25,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. 2023-02-09 10:00:25,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. after waiting 0 ms 2023-02-09 10:00:25,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. 2023-02-09 10:00:25,164 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 958c765f00213055f1a0dc052434ad89 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-09 10:00:25,176 INFO [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(1065): Closing user regions 2023-02-09 10:00:25,180 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/namespace/958c765f00213055f1a0dc052434ad89/.tmp/info/59daa637c84b45dba909f06824c61e93 2023-02-09 10:00:25,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/namespace/958c765f00213055f1a0dc052434ad89/.tmp/info/59daa637c84b45dba909f06824c61e93 as hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/namespace/958c765f00213055f1a0dc052434ad89/info/59daa637c84b45dba909f06824c61e93 2023-02-09 10:00:25,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/namespace/958c765f00213055f1a0dc052434ad89/info/59daa637c84b45dba909f06824c61e93, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 10:00:25,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 958c765f00213055f1a0dc052434ad89 in 34ms, sequenceid=6, compaction requested=false 2023-02-09 10:00:25,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/namespace/958c765f00213055f1a0dc052434ad89/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-09 10:00:25,208 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. 2023-02-09 10:00:25,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 958c765f00213055f1a0dc052434ad89: Waiting for close lock at 1675936825164Running coprocessor pre-close hooks at 1675936825164Disabling compacts and flushes for region at 1675936825164Disabling writes for close at 1675936825164Obtaining lock to block concurrent updates at 1675936825164Preparing flush snapshotting stores in 958c765f00213055f1a0dc052434ad89 at 1675936825164Finished memstore snapshotting hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1675936825164Flushing stores of hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. at 1675936825165 (+1 ms)Flushing 958c765f00213055f1a0dc052434ad89/info: creating writer at 1675936825165Flushing 958c765f00213055f1a0dc052434ad89/info: appending metadata at 1675936825169 (+4 ms)Flushing 958c765f00213055f1a0dc052434ad89/info: closing flushed file at 1675936825169Flushing 958c765f00213055f1a0dc052434ad89/info: reopening flushed file at 1675936825190 (+21 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 958c765f00213055f1a0dc052434ad89 in 34ms, sequenceid=6, compaction requested=false at 1675936825199 (+9 ms)Writing region close event to WAL at 1675936825201 (+2 ms)Running coprocessor post-close hooks at 1675936825208 (+7 ms)Closed at 1675936825208 2023-02-09 10:00:25,209 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33909] assignment.AssignmentManager(1094): RegionServer CLOSED 958c765f00213055f1a0dc052434ad89 2023-02-09 10:00:25,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675936824490.958c765f00213055f1a0dc052434ad89. 2023-02-09 10:00:25,265 INFO [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43217,1675936823692' ***** 2023-02-09 10:00:25,267 INFO [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(2310): STOPPED: Exiting; cluster shutdown set and not carrying any regions 2023-02-09 10:00:25,273 INFO [RS:0;jenkins-hbase4:43217] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 10:00:25,273 INFO [RS:0;jenkins-hbase4:43217] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 10:00:25,273 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 10:00:25,273 INFO [RS:0;jenkins-hbase4:43217] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 10:00:25,274 INFO [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:25,274 DEBUG [RS:0;jenkins-hbase4:43217] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x35b38c4f to 127.0.0.1:58825 2023-02-09 10:00:25,274 DEBUG [RS:0;jenkins-hbase4:43217] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:25,274 INFO [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43217,1675936823692; all regions closed. 2023-02-09 10:00:25,277 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 10:00:25,279 DEBUG [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(1084): Waiting on 1588230740 2023-02-09 10:00:25,281 DEBUG [RS:0;jenkins-hbase4:43217] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/oldWALs 2023-02-09 10:00:25,282 INFO [RS:0;jenkins-hbase4:43217] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43217%2C1675936823692:(num 1675936824214) 2023-02-09 10:00:25,282 DEBUG [RS:0;jenkins-hbase4:43217] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:25,282 INFO [RS:0;jenkins-hbase4:43217] regionserver.LeaseManager(133): Closed leases 2023-02-09 10:00:25,282 INFO [RS:0;jenkins-hbase4:43217] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-09 10:00:25,282 INFO [RS:0;jenkins-hbase4:43217] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 10:00:25,282 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 10:00:25,282 INFO [RS:0;jenkins-hbase4:43217] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 10:00:25,282 INFO [RS:0;jenkins-hbase4:43217] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 10:00:25,283 INFO [RS:0;jenkins-hbase4:43217] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43217 2023-02-09 10:00:25,286 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:25,286 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43217,1675936823692 2023-02-09 10:00:25,286 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:25,286 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:25,286 ERROR [Listener at localhost/34395-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3a7f25fa rejected from java.util.concurrent.ThreadPoolExecutor@3e40a66d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:25,286 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:25,289 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43217,1675936823692] 2023-02-09 10:00:25,290 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43217,1675936823692; numProcessing=2 2023-02-09 10:00:25,290 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:25,290 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,43217,1675936823692 znode expired, triggering replicatorRemoved event 2023-02-09 10:00:25,291 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,43217,1675936823692 already deleted, retry=false 2023-02-09 10:00:25,291 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43217,1675936823692 expired; onlineServers=1 2023-02-09 10:00:25,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:25,301 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:25,301 INFO [RS:2;jenkins-hbase4:38697] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,38697,1675936823747; zookeeper connection closed. 2023-02-09 10:00:25,301 ERROR [Listener at localhost/34395-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@52b30122 rejected from java.util.concurrent.ThreadPoolExecutor@295bc597[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 5] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:25,302 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:38697-0x100388662ff0003, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:25,302 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6a7868d0] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6a7868d0 2023-02-09 10:00:25,302 ERROR [Listener at localhost/34395-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2aff958e rejected from java.util.concurrent.ThreadPoolExecutor@295bc597[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 5] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:25,380 INFO [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,35049,1675936823717' ***** 2023-02-09 10:00:25,380 INFO [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(2310): STOPPED: Stopped; only catalog regions remaining online 2023-02-09 10:00:25,381 INFO [RS:1;jenkins-hbase4:35049] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 10:00:25,381 INFO [RS:1;jenkins-hbase4:35049] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 10:00:25,381 INFO [RS:1;jenkins-hbase4:35049] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 10:00:25,381 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 10:00:25,381 INFO [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:25,382 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 10:00:25,382 DEBUG [RS:1;jenkins-hbase4:35049] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3427e4fe to 127.0.0.1:58825 2023-02-09 10:00:25,383 DEBUG [RS:1;jenkins-hbase4:35049] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:25,383 INFO [RS:1;jenkins-hbase4:35049] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 10:00:25,383 INFO [RS:1;jenkins-hbase4:35049] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 10:00:25,383 INFO [RS:1;jenkins-hbase4:35049] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 10:00:25,383 INFO [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-09 10:00:25,383 INFO [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-09 10:00:25,383 DEBUG [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-09 10:00:25,384 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 10:00:25,384 DEBUG [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 10:00:25,384 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 10:00:25,384 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 10:00:25,384 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 10:00:25,384 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 10:00:25,384 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-02-09 10:00:25,403 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/.tmp/info/b42faff921a34f0989aa261bbba322a6 2023-02-09 10:00:25,425 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/.tmp/table/9557a9e2995e43618716a773d3796572 2023-02-09 10:00:25,433 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/.tmp/info/b42faff921a34f0989aa261bbba322a6 as hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/info/b42faff921a34f0989aa261bbba322a6 2023-02-09 10:00:25,440 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/info/b42faff921a34f0989aa261bbba322a6, entries=10, sequenceid=9, filesize=5.9 K 2023-02-09 10:00:25,441 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/.tmp/table/9557a9e2995e43618716a773d3796572 as hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/table/9557a9e2995e43618716a773d3796572 2023-02-09 10:00:25,450 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/table/9557a9e2995e43618716a773d3796572, entries=2, sequenceid=9, filesize=4.7 K 2023-02-09 10:00:25,451 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 67ms, sequenceid=9, compaction requested=false 2023-02-09 10:00:25,458 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-02-09 10:00:25,459 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 10:00:25,459 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 10:00:25,459 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675936825383Running coprocessor pre-close hooks at 1675936825383Disabling compacts and flushes for region at 1675936825383Disabling writes for close at 1675936825384 (+1 ms)Obtaining lock to block concurrent updates at 1675936825384Preparing flush snapshotting stores in 1588230740 at 1675936825384Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=1290, getHeapSize=2912, getOffHeapSize=0, getCellsCount=12 at 1675936825385 (+1 ms)Flushing stores of hbase:meta,,1.1588230740 at 1675936825386 (+1 ms)Flushing 1588230740/info: creating writer at 1675936825386Flushing 1588230740/info: appending metadata at 1675936825389 (+3 ms)Flushing 1588230740/info: closing flushed file at 1675936825389Flushing 1588230740/table: creating writer at 1675936825412 (+23 ms)Flushing 1588230740/table: appending metadata at 1675936825415 (+3 ms)Flushing 1588230740/table: closing flushed file at 1675936825415Flushing 1588230740/info: reopening flushed file at 1675936825434 (+19 ms)Flushing 1588230740/table: reopening flushed file at 1675936825442 (+8 ms)Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 67ms, sequenceid=9, compaction requested=false at 1675936825451 (+9 ms)Writing region close event to WAL at 1675936825454 (+3 ms)Running coprocessor post-close hooks at 1675936825459 (+5 ms)Closed at 1675936825459 2023-02-09 10:00:25,459 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-09 10:00:25,584 INFO [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,35049,1675936823717; all regions closed. 2023-02-09 10:00:25,591 DEBUG [RS:1;jenkins-hbase4:35049] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/oldWALs 2023-02-09 10:00:25,591 INFO [RS:1;jenkins-hbase4:35049] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35049%2C1675936823717.meta:.meta(num 1675936824412) 2023-02-09 10:00:25,596 DEBUG [RS:1;jenkins-hbase4:35049] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/oldWALs 2023-02-09 10:00:25,596 INFO [RS:1;jenkins-hbase4:35049] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35049%2C1675936823717:(num 1675936824225) 2023-02-09 10:00:25,596 DEBUG [RS:1;jenkins-hbase4:35049] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:25,596 INFO [RS:1;jenkins-hbase4:35049] regionserver.LeaseManager(133): Closed leases 2023-02-09 10:00:25,596 INFO [RS:1;jenkins-hbase4:35049] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-09 10:00:25,597 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 10:00:25,597 INFO [RS:1;jenkins-hbase4:35049] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35049 2023-02-09 10:00:25,601 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,35049,1675936823717 2023-02-09 10:00:25,601 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:25,601 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 10:00:25,601 ERROR [Listener at localhost/34395-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5a8b56da rejected from java.util.concurrent.ThreadPoolExecutor@5eb4d953[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:25,602 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:25,602 INFO [RS:0;jenkins-hbase4:43217] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43217,1675936823692; zookeeper connection closed. 2023-02-09 10:00:25,602 ERROR [Listener at localhost/34395-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@9e0d591 rejected from java.util.concurrent.ThreadPoolExecutor@3e40a66d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:25,602 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@170919d6] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@170919d6 2023-02-09 10:00:25,602 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:43217-0x100388662ff0001, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:25,602 ERROR [Listener at localhost/34395-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@713bd54f rejected from java.util.concurrent.ThreadPoolExecutor@3e40a66d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:25,603 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,35049,1675936823717] 2023-02-09 10:00:25,603 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,35049,1675936823717; numProcessing=3 2023-02-09 10:00:25,604 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,35049,1675936823717 already deleted, retry=false 2023-02-09 10:00:25,605 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,35049,1675936823717 expired; onlineServers=0 2023-02-09 10:00:25,605 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33909,1675936823640' ***** 2023-02-09 10:00:25,605 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-09 10:00:25,605 DEBUG [M:0;jenkins-hbase4:33909] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6ddd99ed, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 10:00:25,605 INFO [M:0;jenkins-hbase4:33909] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33909,1675936823640 2023-02-09 10:00:25,605 INFO [M:0;jenkins-hbase4:33909] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33909,1675936823640; all regions closed. 2023-02-09 10:00:25,605 DEBUG [M:0;jenkins-hbase4:33909] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:25,605 DEBUG [M:0;jenkins-hbase4:33909] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-09 10:00:25,605 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-09 10:00:25,605 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675936823953] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675936823953,5,FailOnTimeoutGroup] 2023-02-09 10:00:25,606 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675936823953] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675936823953,5,FailOnTimeoutGroup] 2023-02-09 10:00:25,605 DEBUG [M:0;jenkins-hbase4:33909] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-09 10:00:25,607 INFO [M:0;jenkins-hbase4:33909] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-09 10:00:25,607 INFO [M:0;jenkins-hbase4:33909] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-09 10:00:25,607 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-09 10:00:25,607 INFO [M:0;jenkins-hbase4:33909] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-09 10:00:25,607 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 10:00:25,607 DEBUG [M:0;jenkins-hbase4:33909] master.HMaster(1502): Stopping service threads 2023-02-09 10:00:25,607 INFO [M:0;jenkins-hbase4:33909] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-09 10:00:25,607 ERROR [M:0;jenkins-hbase4:33909] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] 2023-02-09 10:00:25,608 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 10:00:25,608 INFO [M:0;jenkins-hbase4:33909] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 10:00:25,608 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-09 10:00:25,608 DEBUG [M:0;jenkins-hbase4:33909] zookeeper.ZKUtil(398): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-09 10:00:25,608 WARN [M:0;jenkins-hbase4:33909] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-09 10:00:25,608 INFO [M:0;jenkins-hbase4:33909] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 10:00:25,609 INFO [M:0;jenkins-hbase4:33909] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-09 10:00:25,609 DEBUG [M:0;jenkins-hbase4:33909] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 10:00:25,609 INFO [M:0;jenkins-hbase4:33909] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:25,609 DEBUG [M:0;jenkins-hbase4:33909] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:25,609 DEBUG [M:0;jenkins-hbase4:33909] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 10:00:25,609 DEBUG [M:0;jenkins-hbase4:33909] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:25,609 INFO [M:0;jenkins-hbase4:33909] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-02-09 10:00:25,625 INFO [M:0;jenkins-hbase4:33909] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/335e8bc49c57451f92f2d4bb510b6df2 2023-02-09 10:00:25,632 DEBUG [M:0;jenkins-hbase4:33909] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/335e8bc49c57451f92f2d4bb510b6df2 as hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/335e8bc49c57451f92f2d4bb510b6df2 2023-02-09 10:00:25,639 INFO [M:0;jenkins-hbase4:33909] regionserver.HStore(1080): Added hdfs://localhost:37725/user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/335e8bc49c57451f92f2d4bb510b6df2, entries=8, sequenceid=66, filesize=6.3 K 2023-02-09 10:00:25,640 INFO [M:0;jenkins-hbase4:33909] regionserver.HRegion(2947): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 31ms, sequenceid=66, compaction requested=false 2023-02-09 10:00:25,641 INFO [M:0;jenkins-hbase4:33909] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 10:00:25,642 DEBUG [M:0;jenkins-hbase4:33909] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675936825609Disabling compacts and flushes for region at 1675936825609Disabling writes for close at 1675936825609Obtaining lock to block concurrent updates at 1675936825609Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675936825609Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=24646, getHeapSize=30248, getOffHeapSize=0, getCellsCount=71 at 1675936825610 (+1 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675936825610Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675936825611 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675936825614 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675936825615 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675936825633 (+18 ms)Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 31ms, sequenceid=66, compaction requested=false at 1675936825640 (+7 ms)Writing region close event to WAL at 1675936825641 (+1 ms)Closed at 1675936825641 2023-02-09 10:00:25,645 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/adf0d96a-baac-3808-67fc-4833785957d9/MasterData/WALs/jenkins-hbase4.apache.org,33909,1675936823640/jenkins-hbase4.apache.org%2C33909%2C1675936823640.1675936823863 not finished, retry = 0 2023-02-09 10:00:25,704 INFO [Listener at localhost/34395] client.AsyncConnectionImpl(207): Connection has been closed by Listener at localhost/34395. 2023-02-09 10:00:25,704 DEBUG [Listener at localhost/34395] client.AsyncConnectionImpl(232): Call stack: at java.lang.Thread.getStackTrace(Thread.java:1564) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:209) at org.apache.hbase.thirdparty.com.google.common.io.Closeables.close(Closeables.java:79) at org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2.tearDown(TestAsyncClusterAdminApi2.java:75) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2023-02-09 10:00:25,704 DEBUG [Listener at localhost/34395] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:25,705 DEBUG [Listener at localhost/34395] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x206f8695 to 127.0.0.1:58825 2023-02-09 10:00:25,705 INFO [Listener at localhost/34395] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-09 10:00:25,705 DEBUG [Listener at localhost/34395] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x71d9a02f to 127.0.0.1:58825 2023-02-09 10:00:25,705 DEBUG [Listener at localhost/34395] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 10:00:25,706 DEBUG [Listener at localhost/34395] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-09 10:00:25,746 INFO [M:0;jenkins-hbase4:33909] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-09 10:00:25,748 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 10:00:25,749 INFO [M:0;jenkins-hbase4:33909] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33909 2023-02-09 10:00:25,750 DEBUG [M:0;jenkins-hbase4:33909] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,33909,1675936823640 already deleted, retry=false 2023-02-09 10:00:25,776 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-02-09 10:00:25,803 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:25,803 INFO [RS:1;jenkins-hbase4:35049] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,35049,1675936823717; zookeeper connection closed. 2023-02-09 10:00:25,803 ERROR [Listener at localhost/34395-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2ada281e rejected from java.util.concurrent.ThreadPoolExecutor@5eb4d953[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:25,803 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@788e3e66] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@788e3e66 2023-02-09 10:00:25,803 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): regionserver:35049-0x100388662ff0002, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:25,803 ERROR [Listener at localhost/34395-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5261ac57 rejected from java.util.concurrent.ThreadPoolExecutor@5eb4d953[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:25,803 INFO [Listener at localhost/34395] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-09 10:00:25,903 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:25,903 INFO [M:0;jenkins-hbase4:33909] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33909,1675936823640; zookeeper connection closed. 2023-02-09 10:00:25,903 ERROR [Listener at localhost/34395-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5577b5 rejected from java.util.concurrent.ThreadPoolExecutor@46ae367[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 28] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:25,904 DEBUG [Listener at localhost/34395-EventThread] zookeeper.ZKWatcher(600): master:33909-0x100388662ff0000, quorum=127.0.0.1:58825, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 10:00:25,904 WARN [Listener at localhost/34395] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 10:00:25,904 ERROR [Listener at localhost/34395-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5b208784 rejected from java.util.concurrent.ThreadPoolExecutor@46ae367[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 28] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 10:00:25,908 INFO [Listener at localhost/34395] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 10:00:26,013 WARN [BP-1531302797-172.31.14.131-1675936822910 heartbeating to localhost/127.0.0.1:37725] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-09 10:00:26,013 WARN [BP-1531302797-172.31.14.131-1675936822910 heartbeating to localhost/127.0.0.1:37725] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1531302797-172.31.14.131-1675936822910 (Datanode Uuid e39f1817-24ca-4e85-b648-9a6123e8186d) service to localhost/127.0.0.1:37725 2023-02-09 10:00:26,014 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/cluster_588ab5a6-40ec-fa6b-9119-7cf13de98932/dfs/data/data5/current/BP-1531302797-172.31.14.131-1675936822910] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 10:00:26,014 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/cluster_588ab5a6-40ec-fa6b-9119-7cf13de98932/dfs/data/data6/current/BP-1531302797-172.31.14.131-1675936822910] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 10:00:26,015 WARN [Listener at localhost/34395] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 10:00:26,019 INFO [Listener at localhost/34395] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 10:00:26,122 WARN [BP-1531302797-172.31.14.131-1675936822910 heartbeating to localhost/127.0.0.1:37725] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-09 10:00:26,122 WARN [BP-1531302797-172.31.14.131-1675936822910 heartbeating to localhost/127.0.0.1:37725] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1531302797-172.31.14.131-1675936822910 (Datanode Uuid 2875a852-b6e6-4859-9f5d-c8c929854fe4) service to localhost/127.0.0.1:37725 2023-02-09 10:00:26,123 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/cluster_588ab5a6-40ec-fa6b-9119-7cf13de98932/dfs/data/data3/current/BP-1531302797-172.31.14.131-1675936822910] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 10:00:26,124 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/cluster_588ab5a6-40ec-fa6b-9119-7cf13de98932/dfs/data/data4/current/BP-1531302797-172.31.14.131-1675936822910] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 10:00:26,125 WARN [Listener at localhost/34395] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 10:00:26,128 INFO [Listener at localhost/34395] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 10:00:26,233 WARN [BP-1531302797-172.31.14.131-1675936822910 heartbeating to localhost/127.0.0.1:37725] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-09 10:00:26,233 WARN [BP-1531302797-172.31.14.131-1675936822910 heartbeating to localhost/127.0.0.1:37725] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1531302797-172.31.14.131-1675936822910 (Datanode Uuid 9b2c3394-e51a-4ac6-b18d-5e9681c7044f) service to localhost/127.0.0.1:37725 2023-02-09 10:00:26,234 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/cluster_588ab5a6-40ec-fa6b-9119-7cf13de98932/dfs/data/data1/current/BP-1531302797-172.31.14.131-1675936822910] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 10:00:26,234 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a92a509f-56ef-4808-c1ab-21293d359aab/cluster_588ab5a6-40ec-fa6b-9119-7cf13de98932/dfs/data/data2/current/BP-1531302797-172.31.14.131-1675936822910] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 10:00:26,246 INFO [Listener at localhost/34395] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 10:00:26,360 INFO [Listener at localhost/34395] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-09 10:00:26,387 INFO [Listener at localhost/34395] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-02-09 10:00:26,398 INFO [Listener at localhost/34395] hbase.ResourceChecker(175): after: client.TestAsyncClusterAdminApi2#testShutdown Thread=109 (was 77) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:37725 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager$NodeFailoverWorker.run(ReplicationSourceManager.java:703) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:37725 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/34395 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (985377774) connection to localhost/127.0.0.1:37725 from jenkins.hfs.4 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:37725 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (985377774) connection to localhost/127.0.0.1:37725 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (985377774) connection to localhost/127.0.0.1:37725 from jenkins.hfs.5 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:37725 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager$NodeFailoverWorker.run(ReplicationSourceManager.java:703) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (985377774) connection to localhost/127.0.0.1:37725 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: IPC Client (985377774) connection to localhost/127.0.0.1:37725 from jenkins.hfs.3 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) - Thread LEAK? -, OpenFileDescriptor=547 (was 508) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=301 (was 292) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3274 (was 3359)