2023-08-07 13:55:34,532 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff 2023-08-07 13:55:34,540 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/hadoop-log-dir so I do NOT create it in target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b 2023-08-07 13:55:34,541 WARN [main] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/hadoop-log-dir Erasing configuration value by system value. 2023-08-07 13:55:34,541 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/hadoop-tmp-dir so I do NOT create it in target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b 2023-08-07 13:55:34,541 WARN [main] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/hadoop-tmp-dir Erasing configuration value by system value. 2023-08-07 13:55:34,541 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b 2023-08-07 13:55:34,554 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool timeout: 13 mins 2023-08-07 13:55:34,555 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL timeout: 13 mins 2023-08-07 13:55:34,587 INFO [Time-limited test] hbase.ResourceChecker(147): before: replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL#testSyncUpTool Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=231, ProcessCount=171, AvailableMemoryMB=2687 2023-08-07 13:55:34,601 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/hadoop-log-dir so I do NOT create it in target/test-data/cfb829cf-df43-0b60-0995-5375e17ceeec 2023-08-07 13:55:34,601 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/hadoop-log-dir Erasing configuration value by system value. 2023-08-07 13:55:34,601 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/hadoop-tmp-dir so I do NOT create it in target/test-data/cfb829cf-df43-0b60-0995-5375e17ceeec 2023-08-07 13:55:34,602 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/hadoop-tmp-dir Erasing configuration value by system value. 2023-08-07 13:55:34,602 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfb829cf-df43-0b60-0995-5375e17ceeec 2023-08-07 13:55:34,613 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/hadoop-log-dir so I do NOT create it in target/test-data/f07c87f8-3a64-609c-d227-d4f9a1cc17fb 2023-08-07 13:55:34,613 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/hadoop-log-dir Erasing configuration value by system value. 2023-08-07 13:55:34,613 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/hadoop-tmp-dir so I do NOT create it in target/test-data/f07c87f8-3a64-609c-d227-d4f9a1cc17fb 2023-08-07 13:55:34,613 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/hadoop-tmp-dir Erasing configuration value by system value. 2023-08-07 13:55:34,614 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f07c87f8-3a64-609c-d227-d4f9a1cc17fb 2023-08-07 13:55:34,618 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/cluster_d148ea16-da3d-4a0f-a23f-762caf2bedda, deleteOnExit=true 2023-08-07 13:55:34,750 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/cluster_d148ea16-da3d-4a0f-a23f-762caf2bedda/zookeeper_0, clientPort=52488, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/cluster_d148ea16-da3d-4a0f-a23f-762caf2bedda/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/cluster_d148ea16-da3d-4a0f-a23f-762caf2bedda/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-07 13:55:34,765 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=52488 2023-08-07 13:55:34,769 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=2, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-07 13:55:34,769 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-07 13:55:34,769 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/test.cache.data in system properties and HBase conf 2023-08-07 13:55:34,770 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/hadoop.tmp.dir in system properties and HBase conf 2023-08-07 13:55:34,770 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/hadoop.log.dir in system properties and HBase conf 2023-08-07 13:55:34,770 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-07 13:55:34,771 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-07 13:55:34,771 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-07 13:55:34,880 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-08-07 13:55:35,272 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-07 13:55:35,276 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-07 13:55:35,277 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-07 13:55:35,278 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-07 13:55:35,278 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-07 13:55:35,279 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-07 13:55:35,279 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-07 13:55:35,279 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-07 13:55:35,280 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-07 13:55:35,280 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-07 13:55:35,281 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/nfs.dump.dir in system properties and HBase conf 2023-08-07 13:55:35,281 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/java.io.tmpdir in system properties and HBase conf 2023-08-07 13:55:35,281 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-07 13:55:35,282 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-07 13:55:35,282 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-07 13:55:35,768 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-07 13:55:35,773 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-07 13:55:36,095 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-08-07 13:55:36,267 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-08-07 13:55:36,295 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 13:55:36,343 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 13:55:36,385 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/java.io.tmpdir/Jetty_localhost_36709_hdfs____bcf17r/webapp 2023-08-07 13:55:36,552 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36709 2023-08-07 13:55:36,565 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-07 13:55:36,566 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-07 13:55:37,057 WARN [Listener at localhost/39335] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 13:55:37,119 WARN [Listener at localhost/39335] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 13:55:37,137 WARN [Listener at localhost/39335] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 13:55:37,143 INFO [Listener at localhost/39335] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 13:55:37,147 INFO [Listener at localhost/39335] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/java.io.tmpdir/Jetty_localhost_37681_datanode____9i9y5u/webapp 2023-08-07 13:55:37,243 INFO [Listener at localhost/39335] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37681 2023-08-07 13:55:37,551 WARN [Listener at localhost/34931] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 13:55:37,562 WARN [Listener at localhost/34931] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 13:55:37,566 WARN [Listener at localhost/34931] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 13:55:37,567 INFO [Listener at localhost/34931] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 13:55:37,572 INFO [Listener at localhost/34931] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/java.io.tmpdir/Jetty_localhost_45749_datanode____.2p4j3y/webapp 2023-08-07 13:55:37,680 INFO [Listener at localhost/34931] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45749 2023-08-07 13:55:37,688 WARN [Listener at localhost/39609] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 13:55:38,050 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xceb3e3eb8e47b3b: Processing first storage report for DS-8a538a5a-921c-432a-ad90-b37a7c232eae from datanode 4164ac0a-8f80-434d-8841-c564236eb4dc 2023-08-07 13:55:38,052 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xceb3e3eb8e47b3b: from storage DS-8a538a5a-921c-432a-ad90-b37a7c232eae node DatanodeRegistration(127.0.0.1:33541, datanodeUuid=4164ac0a-8f80-434d-8841-c564236eb4dc, infoPort=37259, infoSecurePort=0, ipcPort=39609, storageInfo=lv=-57;cid=testClusterID;nsid=2029477623;c=1691416535853), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-08-07 13:55:38,052 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc1d9702bb910b497: Processing first storage report for DS-babc7b69-da91-489f-ab02-fda897fb6aca from datanode bd1d5e30-8de9-4355-b26e-64688c9d6d85 2023-08-07 13:55:38,052 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc1d9702bb910b497: from storage DS-babc7b69-da91-489f-ab02-fda897fb6aca node DatanodeRegistration(127.0.0.1:46767, datanodeUuid=bd1d5e30-8de9-4355-b26e-64688c9d6d85, infoPort=34433, infoSecurePort=0, ipcPort=34931, storageInfo=lv=-57;cid=testClusterID;nsid=2029477623;c=1691416535853), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 13:55:38,052 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xceb3e3eb8e47b3b: Processing first storage report for DS-28fa304b-448b-410f-a2c1-81038ebfd331 from datanode 4164ac0a-8f80-434d-8841-c564236eb4dc 2023-08-07 13:55:38,053 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xceb3e3eb8e47b3b: from storage DS-28fa304b-448b-410f-a2c1-81038ebfd331 node DatanodeRegistration(127.0.0.1:33541, datanodeUuid=4164ac0a-8f80-434d-8841-c564236eb4dc, infoPort=37259, infoSecurePort=0, ipcPort=39609, storageInfo=lv=-57;cid=testClusterID;nsid=2029477623;c=1691416535853), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-07 13:55:38,053 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc1d9702bb910b497: Processing first storage report for DS-f737a9c5-355e-4066-9a72-7e1f74e37d71 from datanode bd1d5e30-8de9-4355-b26e-64688c9d6d85 2023-08-07 13:55:38,053 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc1d9702bb910b497: from storage DS-f737a9c5-355e-4066-9a72-7e1f74e37d71 node DatanodeRegistration(127.0.0.1:46767, datanodeUuid=bd1d5e30-8de9-4355-b26e-64688c9d6d85, infoPort=34433, infoSecurePort=0, ipcPort=34931, storageInfo=lv=-57;cid=testClusterID;nsid=2029477623;c=1691416535853), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 13:55:38,143 DEBUG [Listener at localhost/39609] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff 2023-08-07 13:55:38,152 INFO [Listener at localhost/39609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:38,154 INFO [Listener at localhost/39609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:38,813 INFO [Listener at localhost/39609] util.FSUtils(471): Created version file at hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a with version=8 2023-08-07 13:55:38,813 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/hbase-staging 2023-08-07 13:55:39,134 INFO [Listener at localhost/39609] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-08-07 13:55:39,592 INFO [Listener at localhost/39609] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 13:55:39,624 INFO [Listener at localhost/39609] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:39,625 INFO [Listener at localhost/39609] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:39,625 INFO [Listener at localhost/39609] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 13:55:39,625 INFO [Listener at localhost/39609] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:39,625 INFO [Listener at localhost/39609] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 13:55:39,770 INFO [Listener at localhost/39609] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 13:55:39,844 DEBUG [Listener at localhost/39609] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-08-07 13:55:39,938 INFO [Listener at localhost/39609] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38803 2023-08-07 13:55:39,949 INFO [Listener at localhost/39609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:39,952 INFO [Listener at localhost/39609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:39,978 INFO [Listener at localhost/39609] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38803 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:40,019 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:388030x0, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:40,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:38803-0x101dfb6a1a10000 connected 2023-08-07 13:55:40,044 DEBUG [Listener at localhost/39609] zookeeper.ZKUtil(164): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-07 13:55:40,045 DEBUG [Listener at localhost/39609] zookeeper.ZKUtil(164): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-07 13:55:40,049 DEBUG [Listener at localhost/39609] zookeeper.ZKUtil(164): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-07 13:55:40,057 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38803 2023-08-07 13:55:40,057 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38803 2023-08-07 13:55:40,057 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38803 2023-08-07 13:55:40,058 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38803 2023-08-07 13:55:40,058 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38803 2023-08-07 13:55:40,063 INFO [Listener at localhost/39609] master.HMaster(444): hbase.rootdir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a, hbase.cluster.distributed=false 2023-08-07 13:55:40,137 INFO [Listener at localhost/39609] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 13:55:40,138 INFO [Listener at localhost/39609] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:40,138 INFO [Listener at localhost/39609] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:40,138 INFO [Listener at localhost/39609] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 13:55:40,138 INFO [Listener at localhost/39609] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:40,139 INFO [Listener at localhost/39609] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 13:55:40,144 INFO [Listener at localhost/39609] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 13:55:40,148 INFO [Listener at localhost/39609] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38873 2023-08-07 13:55:40,150 INFO [Listener at localhost/39609] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 13:55:40,156 DEBUG [Listener at localhost/39609] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 13:55:40,157 INFO [Listener at localhost/39609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:40,159 INFO [Listener at localhost/39609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:40,161 INFO [Listener at localhost/39609] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38873 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:40,165 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:388730x0, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:40,166 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38873-0x101dfb6a1a10001 connected 2023-08-07 13:55:40,166 DEBUG [Listener at localhost/39609] zookeeper.ZKUtil(164): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-07 13:55:40,167 DEBUG [Listener at localhost/39609] zookeeper.ZKUtil(164): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-07 13:55:40,168 DEBUG [Listener at localhost/39609] zookeeper.ZKUtil(164): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-07 13:55:40,168 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38873 2023-08-07 13:55:40,169 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38873 2023-08-07 13:55:40,169 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38873 2023-08-07 13:55:40,170 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38873 2023-08-07 13:55:40,170 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38873 2023-08-07 13:55:40,183 INFO [Listener at localhost/39609] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 13:55:40,183 INFO [Listener at localhost/39609] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:40,183 INFO [Listener at localhost/39609] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:40,183 INFO [Listener at localhost/39609] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 13:55:40,184 INFO [Listener at localhost/39609] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:40,184 INFO [Listener at localhost/39609] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 13:55:40,184 INFO [Listener at localhost/39609] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 13:55:40,186 INFO [Listener at localhost/39609] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46383 2023-08-07 13:55:40,186 INFO [Listener at localhost/39609] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 13:55:40,187 DEBUG [Listener at localhost/39609] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 13:55:40,188 INFO [Listener at localhost/39609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:40,190 INFO [Listener at localhost/39609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:40,192 INFO [Listener at localhost/39609] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46383 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:40,197 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:463830x0, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:40,198 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46383-0x101dfb6a1a10002 connected 2023-08-07 13:55:40,198 DEBUG [Listener at localhost/39609] zookeeper.ZKUtil(164): regionserver:46383-0x101dfb6a1a10002, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-07 13:55:40,199 DEBUG [Listener at localhost/39609] zookeeper.ZKUtil(164): regionserver:46383-0x101dfb6a1a10002, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-07 13:55:40,200 DEBUG [Listener at localhost/39609] zookeeper.ZKUtil(164): regionserver:46383-0x101dfb6a1a10002, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-07 13:55:40,200 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46383 2023-08-07 13:55:40,201 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46383 2023-08-07 13:55:40,201 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46383 2023-08-07 13:55:40,201 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46383 2023-08-07 13:55:40,202 DEBUG [Listener at localhost/39609] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46383 2023-08-07 13:55:40,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,38803,1691416538961 2023-08-07 13:55:40,212 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-08-07 13:55:40,213 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,38803,1691416538961 2023-08-07 13:55:40,235 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-07 13:55:40,235 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-07 13:55:40,235 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:46383-0x101dfb6a1a10002, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-07 13:55:40,235 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:55:40,236 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/master 2023-08-07 13:55:40,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,38803,1691416538961 from backup master directory 2023-08-07 13:55:40,238 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/master 2023-08-07 13:55:40,241 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,38803,1691416538961 2023-08-07 13:55:40,241 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-08-07 13:55:40,242 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 13:55:40,242 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,38803,1691416538961 2023-08-07 13:55:40,245 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-08-07 13:55:40,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-08-07 13:55:40,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/hbase.id with ID: 4d10afbc-00b5-4a8d-bcc6-ae0771b33a8c 2023-08-07 13:55:40,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:40,396 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:55:40,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x0a122ee6 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:40,474 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3ed19049, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:40,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 13:55:40,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-07 13:55:40,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:55:40,513 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:55:40,548 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store-tmp 2023-08-07 13:55:40,581 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:40,582 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 13:55:40,582 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:40,582 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:40,582 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 13:55:40,582 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:40,582 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:40,582 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 13:55:40,584 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/WALs/jenkins-hbase4.apache.org,38803,1691416538961 2023-08-07 13:55:40,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:40,609 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38803%2C1691416538961.jenkins-hbase4.apache.org%2C38803%2C1691416538961.regiongroup-0, suffix=, logDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/WALs/jenkins-hbase4.apache.org,38803,1691416538961, archiveDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/oldWALs, maxLogs=10 2023-08-07 13:55:40,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-08-07 13:55:40,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-08-07 13:55:40,652 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:190) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:195) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:215) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-08-07 13:55:40,686 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:417) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-07 13:55:40,688 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK] 2023-08-07 13:55:40,688 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK] 2023-08-07 13:55:40,695 DEBUG [RS-EventLoopGroup-4-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-07 13:55:40,766 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/WALs/jenkins-hbase4.apache.org,38803,1691416538961/jenkins-hbase4.apache.org%2C38803%2C1691416538961.jenkins-hbase4.apache.org%2C38803%2C1691416538961.regiongroup-0.1691416540619 2023-08-07 13:55:40,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK], DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK]] 2023-08-07 13:55:40,767 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:40,767 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:40,771 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:40,772 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:40,848 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:40,857 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-07 13:55:40,891 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-07 13:55:40,908 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:40,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:40,915 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:40,933 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:40,937 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 13:55:40,939 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69268378, jitterRate=0.03217926621437073}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 13:55:40,939 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 13:55:40,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-07 13:55:40,967 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-07 13:55:40,967 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-07 13:55:40,970 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 13:55:40,973 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-08-07 13:55:41,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 42 msec 2023-08-07 13:55:41,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-07 13:55:41,042 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-07 13:55:41,048 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-07 13:55:41,074 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-07 13:55:41,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-07 13:55:41,079 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-08-07 13:55:41,084 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-07 13:55:41,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-08-07 13:55:41,091 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:55:41,093 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-08-07 13:55:41,093 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-08-07 13:55:41,105 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-08-07 13:55:41,109 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-07 13:55:41,109 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:46383-0x101dfb6a1a10002, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-07 13:55:41,109 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-07 13:55:41,109 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:55:41,110 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,38803,1691416538961, sessionid=0x101dfb6a1a10000, setting cluster-up flag (Was=false) 2023-08-07 13:55:41,126 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:55:41,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-08-07 13:55:41,134 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38803,1691416538961 2023-08-07 13:55:41,139 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:55:41,145 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-08-07 13:55:41,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38803,1691416538961 2023-08-07 13:55:41,149 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.hbase-snapshot/.tmp 2023-08-07 13:55:41,206 INFO [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(951): ClusterId : 4d10afbc-00b5-4a8d-bcc6-ae0771b33a8c 2023-08-07 13:55:41,206 INFO [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(951): ClusterId : 4d10afbc-00b5-4a8d-bcc6-ae0771b33a8c 2023-08-07 13:55:41,211 DEBUG [RS:1;jenkins-hbase4:46383] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 13:55:41,211 DEBUG [RS:0;jenkins-hbase4:38873] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 13:55:41,218 DEBUG [RS:1;jenkins-hbase4:46383] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 13:55:41,218 DEBUG [RS:0;jenkins-hbase4:38873] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 13:55:41,218 DEBUG [RS:1;jenkins-hbase4:46383] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 13:55:41,218 DEBUG [RS:0;jenkins-hbase4:38873] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 13:55:41,223 DEBUG [RS:1;jenkins-hbase4:46383] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 13:55:41,223 DEBUG [RS:0;jenkins-hbase4:38873] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 13:55:41,224 DEBUG [RS:1;jenkins-hbase4:46383] zookeeper.ReadOnlyZKClient(139): Connect 0x1d3e587a to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:41,224 DEBUG [RS:0;jenkins-hbase4:38873] zookeeper.ReadOnlyZKClient(139): Connect 0x29ed6361 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:41,233 DEBUG [RS:1;jenkins-hbase4:46383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@13f9a5ad, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:41,234 DEBUG [RS:1;jenkins-hbase4:46383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@17c6330a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 13:55:41,242 DEBUG [RS:0;jenkins-hbase4:38873] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@272a982c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:41,243 DEBUG [RS:0;jenkins-hbase4:38873] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@23150811, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 13:55:41,266 DEBUG [RS:0;jenkins-hbase4:38873] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:38873 2023-08-07 13:55:41,266 DEBUG [RS:1;jenkins-hbase4:46383] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:46383 2023-08-07 13:55:41,273 INFO [RS:1;jenkins-hbase4:46383] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 13:55:41,273 INFO [RS:0;jenkins-hbase4:38873] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 13:55:41,274 INFO [RS:1;jenkins-hbase4:46383] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 13:55:41,274 INFO [RS:0;jenkins-hbase4:38873] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 13:55:41,274 DEBUG [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 13:55:41,274 DEBUG [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 13:55:41,278 INFO [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38803,1691416538961 with isa=jenkins-hbase4.apache.org/172.31.14.131:46383, startcode=1691416540182 2023-08-07 13:55:41,278 INFO [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38803,1691416538961 with isa=jenkins-hbase4.apache.org/172.31.14.131:38873, startcode=1691416540136 2023-08-07 13:55:41,285 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-07 13:55:41,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:55:41,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:55:41,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:55:41,300 DEBUG [RS:1;jenkins-hbase4:46383] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 13:55:41,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:55:41,300 DEBUG [RS:0;jenkins-hbase4:38873] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 13:55:41,301 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-07 13:55:41,301 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,301 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 13:55:41,301 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691416571312 2023-08-07 13:55:41,314 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-07 13:55:41,316 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-07 13:55:41,316 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-07 13:55:41,322 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-07 13:55:41,326 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-07 13:55:41,338 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-07 13:55:41,338 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-07 13:55:41,339 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-07 13:55:41,339 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-07 13:55:41,341 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,343 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-07 13:55:41,345 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-07 13:55:41,345 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-07 13:55:41,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-07 13:55:41,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-07 13:55:41,351 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416541351,5,FailOnTimeoutGroup] 2023-08-07 13:55:41,351 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416541351,5,FailOnTimeoutGroup] 2023-08-07 13:55:41,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,355 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-07 13:55:41,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,374 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48643, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 13:55:41,374 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56193, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 13:55:41,376 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-07 13:55:41,377 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-07 13:55:41,378 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a 2023-08-07 13:55:41,392 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38803] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:55:41,406 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:41,407 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38803] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:55:41,410 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 13:55:41,413 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/info 2023-08-07 13:55:41,414 DEBUG [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a 2023-08-07 13:55:41,414 DEBUG [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a 2023-08-07 13:55:41,414 DEBUG [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39335 2023-08-07 13:55:41,414 DEBUG [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39335 2023-08-07 13:55:41,414 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 13:55:41,415 DEBUG [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 13:55:41,414 DEBUG [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 13:55:41,416 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:41,416 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 13:55:41,419 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/rep_barrier 2023-08-07 13:55:41,420 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 13:55:41,422 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:41,422 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 13:55:41,423 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-07 13:55:41,425 DEBUG [RS:1;jenkins-hbase4:46383] zookeeper.ZKUtil(162): regionserver:46383-0x101dfb6a1a10002, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:55:41,425 WARN [RS:1;jenkins-hbase4:46383] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 13:55:41,425 INFO [RS:1;jenkins-hbase4:46383] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:55:41,426 INFO [RS:1;jenkins-hbase4:46383] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:55:41,427 DEBUG [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:55:41,427 DEBUG [RS:0;jenkins-hbase4:38873] zookeeper.ZKUtil(162): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:55:41,427 WARN [RS:0;jenkins-hbase4:38873] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 13:55:41,427 INFO [RS:0;jenkins-hbase4:38873] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:55:41,427 INFO [RS:0;jenkins-hbase4:38873] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:55:41,428 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/table 2023-08-07 13:55:41,428 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38873,1691416540136] 2023-08-07 13:55:41,428 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46383,1691416540182] 2023-08-07 13:55:41,428 DEBUG [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:55:41,430 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 13:55:41,431 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:41,435 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740 2023-08-07 13:55:41,435 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740 2023-08-07 13:55:41,440 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-07 13:55:41,441 DEBUG [RS:1;jenkins-hbase4:46383] zookeeper.ZKUtil(162): regionserver:46383-0x101dfb6a1a10002, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:55:41,441 DEBUG [RS:0;jenkins-hbase4:38873] zookeeper.ZKUtil(162): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:55:41,442 DEBUG [RS:1;jenkins-hbase4:46383] zookeeper.ZKUtil(162): regionserver:46383-0x101dfb6a1a10002, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:55:41,442 DEBUG [RS:0;jenkins-hbase4:38873] zookeeper.ZKUtil(162): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:55:41,443 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 13:55:41,447 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 13:55:41,448 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75163401, jitterRate=0.12002195417881012}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-07 13:55:41,448 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 13:55:41,448 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 13:55:41,448 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 13:55:41,448 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 13:55:41,448 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 13:55:41,448 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 13:55:41,449 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 13:55:41,449 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 13:55:41,455 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-07 13:55:41,455 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-07 13:55:41,456 DEBUG [RS:0;jenkins-hbase4:38873] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-07 13:55:41,456 DEBUG [RS:1;jenkins-hbase4:46383] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-07 13:55:41,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-07 13:55:41,466 INFO [RS:1;jenkins-hbase4:46383] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 13:55:41,466 INFO [RS:0;jenkins-hbase4:38873] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 13:55:41,477 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-07 13:55:41,481 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-07 13:55:41,492 INFO [RS:0;jenkins-hbase4:38873] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 13:55:41,492 INFO [RS:1;jenkins-hbase4:46383] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 13:55:41,496 INFO [RS:1;jenkins-hbase4:46383] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 13:55:41,496 INFO [RS:0;jenkins-hbase4:38873] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 13:55:41,496 INFO [RS:1;jenkins-hbase4:46383] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,496 INFO [RS:0;jenkins-hbase4:38873] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,497 INFO [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 13:55:41,497 INFO [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 13:55:41,506 INFO [RS:1;jenkins-hbase4:46383] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,506 INFO [RS:0;jenkins-hbase4:38873] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,506 DEBUG [RS:1;jenkins-hbase4:46383] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,506 DEBUG [RS:0;jenkins-hbase4:38873] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,506 DEBUG [RS:1;jenkins-hbase4:46383] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,506 DEBUG [RS:0;jenkins-hbase4:38873] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,506 DEBUG [RS:1;jenkins-hbase4:46383] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,506 DEBUG [RS:0;jenkins-hbase4:38873] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,506 DEBUG [RS:1;jenkins-hbase4:46383] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,507 DEBUG [RS:0;jenkins-hbase4:38873] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,507 DEBUG [RS:1;jenkins-hbase4:46383] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,507 DEBUG [RS:0;jenkins-hbase4:38873] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,507 DEBUG [RS:1;jenkins-hbase4:46383] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 13:55:41,507 DEBUG [RS:0;jenkins-hbase4:38873] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 13:55:41,507 DEBUG [RS:1;jenkins-hbase4:46383] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,507 DEBUG [RS:0;jenkins-hbase4:38873] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,507 DEBUG [RS:1;jenkins-hbase4:46383] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,507 DEBUG [RS:0;jenkins-hbase4:38873] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,507 DEBUG [RS:1;jenkins-hbase4:46383] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,507 DEBUG [RS:0;jenkins-hbase4:38873] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,507 DEBUG [RS:1;jenkins-hbase4:46383] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,507 DEBUG [RS:0;jenkins-hbase4:38873] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:41,508 INFO [RS:0;jenkins-hbase4:38873] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,509 INFO [RS:0;jenkins-hbase4:38873] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,509 INFO [RS:0;jenkins-hbase4:38873] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,510 INFO [RS:1;jenkins-hbase4:46383] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,510 INFO [RS:1;jenkins-hbase4:46383] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,510 INFO [RS:1;jenkins-hbase4:46383] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,526 INFO [RS:0;jenkins-hbase4:38873] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 13:55:41,526 INFO [RS:1;jenkins-hbase4:46383] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 13:55:41,529 INFO [RS:0;jenkins-hbase4:38873] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38873,1691416540136-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,529 INFO [RS:1;jenkins-hbase4:46383] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46383,1691416540182-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:41,546 INFO [RS:1;jenkins-hbase4:46383] regionserver.Replication(203): jenkins-hbase4.apache.org,46383,1691416540182 started 2023-08-07 13:55:41,547 INFO [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46383,1691416540182, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46383, sessionid=0x101dfb6a1a10002 2023-08-07 13:55:41,547 DEBUG [RS:1;jenkins-hbase4:46383] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 13:55:41,547 DEBUG [RS:1;jenkins-hbase4:46383] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:55:41,547 DEBUG [RS:1;jenkins-hbase4:46383] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46383,1691416540182' 2023-08-07 13:55:41,547 DEBUG [RS:1;jenkins-hbase4:46383] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-08-07 13:55:41,548 DEBUG [RS:1;jenkins-hbase4:46383] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-08-07 13:55:41,549 DEBUG [RS:1;jenkins-hbase4:46383] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 13:55:41,549 DEBUG [RS:1;jenkins-hbase4:46383] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 13:55:41,549 DEBUG [RS:1;jenkins-hbase4:46383] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:55:41,549 DEBUG [RS:1;jenkins-hbase4:46383] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46383,1691416540182' 2023-08-07 13:55:41,549 DEBUG [RS:1;jenkins-hbase4:46383] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-08-07 13:55:41,549 DEBUG [RS:1;jenkins-hbase4:46383] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-08-07 13:55:41,549 INFO [RS:0;jenkins-hbase4:38873] regionserver.Replication(203): jenkins-hbase4.apache.org,38873,1691416540136 started 2023-08-07 13:55:41,549 INFO [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,38873,1691416540136, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38873, sessionid=0x101dfb6a1a10001 2023-08-07 13:55:41,550 DEBUG [RS:0;jenkins-hbase4:38873] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 13:55:41,550 DEBUG [RS:0;jenkins-hbase4:38873] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:55:41,550 DEBUG [RS:0;jenkins-hbase4:38873] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38873,1691416540136' 2023-08-07 13:55:41,551 DEBUG [RS:0;jenkins-hbase4:38873] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-08-07 13:55:41,551 DEBUG [RS:1;jenkins-hbase4:46383] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 13:55:41,551 INFO [RS:1;jenkins-hbase4:46383] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 13:55:41,551 INFO [RS:1;jenkins-hbase4:46383] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 13:55:41,551 DEBUG [RS:0;jenkins-hbase4:38873] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-08-07 13:55:41,552 DEBUG [RS:0;jenkins-hbase4:38873] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 13:55:41,552 DEBUG [RS:0;jenkins-hbase4:38873] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 13:55:41,552 DEBUG [RS:0;jenkins-hbase4:38873] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:55:41,552 DEBUG [RS:0;jenkins-hbase4:38873] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38873,1691416540136' 2023-08-07 13:55:41,552 DEBUG [RS:0;jenkins-hbase4:38873] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-08-07 13:55:41,552 DEBUG [RS:0;jenkins-hbase4:38873] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-08-07 13:55:41,553 DEBUG [RS:0;jenkins-hbase4:38873] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 13:55:41,553 INFO [RS:0;jenkins-hbase4:38873] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 13:55:41,553 INFO [RS:0;jenkins-hbase4:38873] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 13:55:41,633 DEBUG [jenkins-hbase4:38803] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=2, allServersCount=2 2023-08-07 13:55:41,637 DEBUG [jenkins-hbase4:38803] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 13:55:41,645 DEBUG [jenkins-hbase4:38803] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 13:55:41,645 DEBUG [jenkins-hbase4:38803] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 13:55:41,645 DEBUG [jenkins-hbase4:38803] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 13:55:41,648 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38873,1691416540136, state=OPENING 2023-08-07 13:55:41,660 INFO [RS:0;jenkins-hbase4:38873] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:41,660 INFO [RS:1;jenkins-hbase4:46383] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:41,660 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-07 13:55:41,663 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:55:41,664 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-08-07 13:55:41,664 INFO [RS:1;jenkins-hbase4:46383] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0, suffix=, logDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182, archiveDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs, maxLogs=10 2023-08-07 13:55:41,664 INFO [RS:0;jenkins-hbase4:38873] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0, suffix=, logDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136, archiveDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs, maxLogs=10 2023-08-07 13:55:41,670 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38873,1691416540136}] 2023-08-07 13:55:41,709 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK] 2023-08-07 13:55:41,711 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK] 2023-08-07 13:55:41,734 INFO [RS:0;jenkins-hbase4:38873] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0.1691416541672 2023-08-07 13:55:41,739 DEBUG [RS:0;jenkins-hbase4:38873] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK], DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK]] 2023-08-07 13:55:41,744 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK] 2023-08-07 13:55:41,744 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK] 2023-08-07 13:55:41,753 INFO [RS:1;jenkins-hbase4:46383] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0.1691416541682 2023-08-07 13:55:41,753 DEBUG [RS:1;jenkins-hbase4:46383] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK], DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK]] 2023-08-07 13:55:41,888 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:55:41,890 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 13:55:41,894 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49794, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 13:55:41,906 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-07 13:55:41,907 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:55:41,907 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:55:41,907 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:41,911 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38873%2C1691416540136.meta, suffix=.meta, logDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136, archiveDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs, maxLogs=10 2023-08-07 13:55:41,930 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK] 2023-08-07 13:55:41,930 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK] 2023-08-07 13:55:41,935 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.meta.1691416541913.meta 2023-08-07 13:55:41,936 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK], DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK]] 2023-08-07 13:55:41,937 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:41,939 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-07 13:55:41,957 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-07 13:55:41,990 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-07 13:55:41,996 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-07 13:55:41,996 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:41,996 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-07 13:55:41,996 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-07 13:55:42,000 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 13:55:42,002 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/info 2023-08-07 13:55:42,002 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/info 2023-08-07 13:55:42,003 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 13:55:42,003 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:42,004 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 13:55:42,005 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/rep_barrier 2023-08-07 13:55:42,005 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/rep_barrier 2023-08-07 13:55:42,006 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 13:55:42,007 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:42,007 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 13:55:42,008 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/table 2023-08-07 13:55:42,009 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/table 2023-08-07 13:55:42,009 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 13:55:42,010 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:42,012 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740 2023-08-07 13:55:42,015 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740 2023-08-07 13:55:42,018 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-07 13:55:42,021 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 13:55:42,023 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61897060, jitterRate=-0.07766193151473999}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-07 13:55:42,024 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 13:55:42,039 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1691416541880 2023-08-07 13:55:42,062 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-07 13:55:42,063 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-07 13:55:42,063 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38873,1691416540136, state=OPEN 2023-08-07 13:55:42,067 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-08-07 13:55:42,067 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-08-07 13:55:42,071 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-07 13:55:42,071 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38873,1691416540136 in 398 msec 2023-08-07 13:55:42,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-07 13:55:42,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 608 msec 2023-08-07 13:55:42,082 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 885 msec 2023-08-07 13:55:42,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691416542082, completionTime=-1 2023-08-07 13:55:42,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=2; waited=0ms, expected min=2 server(s), max=2 server(s), master is running 2023-08-07 13:55:42,083 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 13:55:42,150 DEBUG [hconnection-0x415ca050-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:42,153 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49798, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:55:42,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=2 2023-08-07 13:55:42,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691416602173 2023-08-07 13:55:42,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691416662173 2023-08-07 13:55:42,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 90 msec 2023-08-07 13:55:42,198 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38803,1691416538961-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:42,199 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38803,1691416538961-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:42,199 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38803,1691416538961-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:42,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38803, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:42,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:42,208 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-07 13:55:42,217 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-07 13:55:42,218 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-07 13:55:42,228 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-07 13:55:42,231 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 13:55:42,234 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 13:55:42,257 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:55:42,259 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d empty. 2023-08-07 13:55:42,260 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:55:42,260 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-07 13:55:42,290 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-07 13:55:42,292 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => ee0eecdf41bc57a826e75999fe370d6d, NAME => 'hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp 2023-08-07 13:55:42,310 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:42,311 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing ee0eecdf41bc57a826e75999fe370d6d, disabling compactions & flushes 2023-08-07 13:55:42,311 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:55:42,311 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:55:42,311 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. after waiting 0 ms 2023-08-07 13:55:42,311 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:55:42,311 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:55:42,311 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for ee0eecdf41bc57a826e75999fe370d6d: 2023-08-07 13:55:42,315 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 13:55:42,330 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691416542318"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691416542318"}]},"ts":"1691416542318"} 2023-08-07 13:55:42,358 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 13:55:42,360 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 13:55:42,365 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691416542361"}]},"ts":"1691416542361"} 2023-08-07 13:55:42,370 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-07 13:55:42,374 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 13:55:42,377 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 13:55:42,377 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 13:55:42,377 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 13:55:42,380 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ee0eecdf41bc57a826e75999fe370d6d, ASSIGN}] 2023-08-07 13:55:42,383 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ee0eecdf41bc57a826e75999fe370d6d, ASSIGN 2023-08-07 13:55:42,384 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=ee0eecdf41bc57a826e75999fe370d6d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46383,1691416540182; forceNewPlan=false, retain=false 2023-08-07 13:55:42,536 INFO [jenkins-hbase4:38803] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 13:55:42,537 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ee0eecdf41bc57a826e75999fe370d6d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:55:42,538 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691416542537"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416542537"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416542537"}]},"ts":"1691416542537"} 2023-08-07 13:55:42,542 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure ee0eecdf41bc57a826e75999fe370d6d, server=jenkins-hbase4.apache.org,46383,1691416540182}] 2023-08-07 13:55:42,697 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:55:42,698 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 13:55:42,700 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49662, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 13:55:42,707 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:55:42,708 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:42,711 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1, suffix=, logDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182, archiveDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs, maxLogs=10 2023-08-07 13:55:42,732 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK] 2023-08-07 13:55:42,732 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK] 2023-08-07 13:55:42,738 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1.1691416542713 2023-08-07 13:55:42,738 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK], DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK]] 2023-08-07 13:55:42,739 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ee0eecdf41bc57a826e75999fe370d6d, NAME => 'hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:42,740 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:55:42,740 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:42,740 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:55:42,740 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:55:42,742 INFO [StoreOpener-ee0eecdf41bc57a826e75999fe370d6d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:55:42,744 DEBUG [StoreOpener-ee0eecdf41bc57a826e75999fe370d6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d/info 2023-08-07 13:55:42,745 DEBUG [StoreOpener-ee0eecdf41bc57a826e75999fe370d6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d/info 2023-08-07 13:55:42,745 INFO [StoreOpener-ee0eecdf41bc57a826e75999fe370d6d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee0eecdf41bc57a826e75999fe370d6d columnFamilyName info 2023-08-07 13:55:42,746 INFO [StoreOpener-ee0eecdf41bc57a826e75999fe370d6d-1] regionserver.HStore(310): Store=ee0eecdf41bc57a826e75999fe370d6d/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:42,748 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:55:42,749 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:55:42,753 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:55:42,757 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 13:55:42,758 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ee0eecdf41bc57a826e75999fe370d6d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71714671, jitterRate=0.06863187253475189}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 13:55:42,758 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ee0eecdf41bc57a826e75999fe370d6d: 2023-08-07 13:55:42,760 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d., pid=6, masterSystemTime=1691416542697 2023-08-07 13:55:42,765 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:55:42,766 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:55:42,768 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ee0eecdf41bc57a826e75999fe370d6d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:55:42,768 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691416542767"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416542767"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416542767"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416542767"}]},"ts":"1691416542767"} 2023-08-07 13:55:42,777 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-07 13:55:42,777 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure ee0eecdf41bc57a826e75999fe370d6d, server=jenkins-hbase4.apache.org,46383,1691416540182 in 231 msec 2023-08-07 13:55:42,780 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-07 13:55:42,781 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=ee0eecdf41bc57a826e75999fe370d6d, ASSIGN in 397 msec 2023-08-07 13:55:42,782 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 13:55:42,782 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691416542782"}]},"ts":"1691416542782"} 2023-08-07 13:55:42,785 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-07 13:55:42,789 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 13:55:42,791 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 570 msec 2023-08-07 13:55:42,831 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/namespace 2023-08-07 13:55:42,833 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/namespace 2023-08-07 13:55:42,833 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:55:42,855 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:42,858 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49668, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:55:42,871 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-07 13:55:42,892 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-08-07 13:55:42,898 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 34 msec 2023-08-07 13:55:42,905 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-07 13:55:42,916 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-08-07 13:55:42,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-08-07 13:55:42,932 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-08-07 13:55:42,935 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-08-07 13:55:42,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.694sec 2023-08-07 13:55:42,939 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-07 13:55:42,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-07 13:55:42,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-07 13:55:42,942 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38803,1691416538961-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-07 13:55:42,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38803,1691416538961-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-07 13:55:42,952 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-07 13:55:43,012 DEBUG [Listener at localhost/39609] zookeeper.ReadOnlyZKClient(139): Connect 0x1aa5818b to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:43,019 DEBUG [Listener at localhost/39609] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5438d971, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:43,034 DEBUG [hconnection-0x52b2e3b9-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:43,045 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49800, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:55:43,055 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,38803,1691416538961 2023-08-07 13:55:43,055 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=4, rsPorts=, rsClass=null, numDataNodes=4, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-07 13:55:43,055 INFO [Listener at localhost/39609] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/cluster_4d37a1ca-6c9d-592f-955d-f460037903b1, deleteOnExit=true 2023-08-07 13:55:43,055 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-07 13:55:43,068 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/test.cache.data in system properties and HBase conf 2023-08-07 13:55:43,068 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/hadoop.tmp.dir in system properties and HBase conf 2023-08-07 13:55:43,068 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/hadoop.log.dir in system properties and HBase conf 2023-08-07 13:55:43,068 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-07 13:55:43,069 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-07 13:55:43,069 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-07 13:55:43,069 DEBUG [Listener at localhost/39609] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-07 13:55:43,069 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-07 13:55:43,070 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-07 13:55:43,070 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-07 13:55:43,071 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-07 13:55:43,071 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-07 13:55:43,071 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-07 13:55:43,071 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-07 13:55:43,072 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-07 13:55:43,072 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-07 13:55:43,072 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/nfs.dump.dir in system properties and HBase conf 2023-08-07 13:55:43,072 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/java.io.tmpdir in system properties and HBase conf 2023-08-07 13:55:43,072 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-07 13:55:43,073 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-07 13:55:43,073 INFO [Listener at localhost/39609] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-07 13:55:43,108 WARN [Listener at localhost/39609] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-07 13:55:43,109 WARN [Listener at localhost/39609] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-07 13:55:43,152 WARN [Listener at localhost/39609] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 13:55:43,155 INFO [Listener at localhost/39609] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 13:55:43,159 INFO [Listener at localhost/39609] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/java.io.tmpdir/Jetty_localhost_37481_hdfs____vnansj/webapp 2023-08-07 13:55:43,254 INFO [Listener at localhost/39609] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37481 2023-08-07 13:55:43,258 WARN [Listener at localhost/39609] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-07 13:55:43,259 WARN [Listener at localhost/39609] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-07 13:55:43,302 WARN [Listener at localhost/45993] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 13:55:43,319 WARN [Listener at localhost/45993] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 13:55:43,322 WARN [Listener at localhost/45993] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 13:55:43,323 INFO [Listener at localhost/45993] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 13:55:43,327 INFO [Listener at localhost/45993] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/java.io.tmpdir/Jetty_localhost_33903_datanode____x75u2r/webapp 2023-08-07 13:55:43,421 INFO [Listener at localhost/45993] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33903 2023-08-07 13:55:43,431 WARN [Listener at localhost/44033] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 13:55:43,450 WARN [Listener at localhost/44033] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 13:55:43,453 WARN [Listener at localhost/44033] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 13:55:43,455 INFO [Listener at localhost/44033] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 13:55:43,464 INFO [Listener at localhost/44033] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/java.io.tmpdir/Jetty_localhost_42735_datanode____.kmmkn2/webapp 2023-08-07 13:55:43,554 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x2bc101e05242f68d: Processing first storage report for DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a from datanode 9ff4caa8-b296-41e4-aa7c-8583e50bff2d 2023-08-07 13:55:43,554 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x2bc101e05242f68d: from storage DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a node DatanodeRegistration(127.0.0.1:40529, datanodeUuid=9ff4caa8-b296-41e4-aa7c-8583e50bff2d, infoPort=33647, infoSecurePort=0, ipcPort=44033, storageInfo=lv=-57;cid=testClusterID;nsid=43455097;c=1691416543114), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 13:55:43,555 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x2bc101e05242f68d: Processing first storage report for DS-f563675e-cfcf-4761-acfa-1e8500a845b7 from datanode 9ff4caa8-b296-41e4-aa7c-8583e50bff2d 2023-08-07 13:55:43,555 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x2bc101e05242f68d: from storage DS-f563675e-cfcf-4761-acfa-1e8500a845b7 node DatanodeRegistration(127.0.0.1:40529, datanodeUuid=9ff4caa8-b296-41e4-aa7c-8583e50bff2d, infoPort=33647, infoSecurePort=0, ipcPort=44033, storageInfo=lv=-57;cid=testClusterID;nsid=43455097;c=1691416543114), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 13:55:43,579 INFO [Listener at localhost/44033] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42735 2023-08-07 13:55:43,588 WARN [Listener at localhost/39103] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 13:55:43,607 WARN [Listener at localhost/39103] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 13:55:43,610 WARN [Listener at localhost/39103] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 13:55:43,611 INFO [Listener at localhost/39103] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 13:55:43,616 INFO [Listener at localhost/39103] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/java.io.tmpdir/Jetty_localhost_33083_datanode____q1mfxu/webapp 2023-08-07 13:55:43,703 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe923f5d32504ca96: Processing first storage report for DS-489b8abe-c957-46e6-ace5-75708cf8539b from datanode cf80ca65-cded-43cf-8fd5-044c1877768f 2023-08-07 13:55:43,703 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe923f5d32504ca96: from storage DS-489b8abe-c957-46e6-ace5-75708cf8539b node DatanodeRegistration(127.0.0.1:32865, datanodeUuid=cf80ca65-cded-43cf-8fd5-044c1877768f, infoPort=39217, infoSecurePort=0, ipcPort=39103, storageInfo=lv=-57;cid=testClusterID;nsid=43455097;c=1691416543114), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 13:55:43,703 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe923f5d32504ca96: Processing first storage report for DS-7d3fff90-7485-4803-9f0e-e0920f9a0fcf from datanode cf80ca65-cded-43cf-8fd5-044c1877768f 2023-08-07 13:55:43,703 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe923f5d32504ca96: from storage DS-7d3fff90-7485-4803-9f0e-e0920f9a0fcf node DatanodeRegistration(127.0.0.1:32865, datanodeUuid=cf80ca65-cded-43cf-8fd5-044c1877768f, infoPort=39217, infoSecurePort=0, ipcPort=39103, storageInfo=lv=-57;cid=testClusterID;nsid=43455097;c=1691416543114), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 13:55:43,726 INFO [Listener at localhost/39103] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33083 2023-08-07 13:55:43,736 WARN [Listener at localhost/39027] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 13:55:43,755 WARN [Listener at localhost/39027] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-07 13:55:43,758 WARN [Listener at localhost/39027] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-07 13:55:43,759 INFO [Listener at localhost/39027] log.Slf4jLog(67): jetty-6.1.26 2023-08-07 13:55:43,765 INFO [Listener at localhost/39027] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/java.io.tmpdir/Jetty_localhost_36833_datanode____svumyk/webapp 2023-08-07 13:55:43,866 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc0dd8463edf91194: Processing first storage report for DS-199fa256-8376-4ed4-8432-68ef01676f76 from datanode bc8e0135-89c1-470e-8efa-86ea0613a4e1 2023-08-07 13:55:43,867 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc0dd8463edf91194: from storage DS-199fa256-8376-4ed4-8432-68ef01676f76 node DatanodeRegistration(127.0.0.1:35779, datanodeUuid=bc8e0135-89c1-470e-8efa-86ea0613a4e1, infoPort=36693, infoSecurePort=0, ipcPort=39027, storageInfo=lv=-57;cid=testClusterID;nsid=43455097;c=1691416543114), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-07 13:55:43,867 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc0dd8463edf91194: Processing first storage report for DS-d8e94705-1001-415b-98e4-bcb32d4bf12f from datanode bc8e0135-89c1-470e-8efa-86ea0613a4e1 2023-08-07 13:55:43,867 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc0dd8463edf91194: from storage DS-d8e94705-1001-415b-98e4-bcb32d4bf12f node DatanodeRegistration(127.0.0.1:35779, datanodeUuid=bc8e0135-89c1-470e-8efa-86ea0613a4e1, infoPort=36693, infoSecurePort=0, ipcPort=39027, storageInfo=lv=-57;cid=testClusterID;nsid=43455097;c=1691416543114), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 13:55:43,890 INFO [Listener at localhost/39027] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36833 2023-08-07 13:55:43,900 WARN [Listener at localhost/41247] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-07 13:55:44,021 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x493cebe0e3291044: Processing first storage report for DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265 from datanode 6efc2cb2-167f-4008-b6ed-09aaade58b38 2023-08-07 13:55:44,021 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x493cebe0e3291044: from storage DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265 node DatanodeRegistration(127.0.0.1:37685, datanodeUuid=6efc2cb2-167f-4008-b6ed-09aaade58b38, infoPort=41631, infoSecurePort=0, ipcPort=41247, storageInfo=lv=-57;cid=testClusterID;nsid=43455097;c=1691416543114), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 13:55:44,021 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x493cebe0e3291044: Processing first storage report for DS-b8426f59-667b-4809-bf43-e84ac1b88031 from datanode 6efc2cb2-167f-4008-b6ed-09aaade58b38 2023-08-07 13:55:44,021 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x493cebe0e3291044: from storage DS-b8426f59-667b-4809-bf43-e84ac1b88031 node DatanodeRegistration(127.0.0.1:37685, datanodeUuid=6efc2cb2-167f-4008-b6ed-09aaade58b38, infoPort=41631, infoSecurePort=0, ipcPort=41247, storageInfo=lv=-57;cid=testClusterID;nsid=43455097;c=1691416543114), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-07 13:55:44,113 DEBUG [Listener at localhost/41247] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b 2023-08-07 13:55:44,113 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:44,114 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:44,157 INFO [Listener at localhost/41247] util.FSUtils(471): Created version file at hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2 with version=8 2023-08-07 13:55:44,158 INFO [Listener at localhost/41247] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/hbase-staging 2023-08-07 13:55:44,160 INFO [Listener at localhost/41247] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-08-07 13:55:44,160 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,160 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,161 INFO [Listener at localhost/41247] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 13:55:44,161 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,161 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 13:55:44,161 INFO [Listener at localhost/41247] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 13:55:44,166 INFO [Listener at localhost/41247] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45527 2023-08-07 13:55:44,167 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:44,169 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:44,170 INFO [Listener at localhost/41247] zookeeper.RecoverableZooKeeper(93): Process identifier=master:45527 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:44,177 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:455270x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:44,184 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:45527-0x101dfb6a1a10007 connected 2023-08-07 13:55:44,196 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-07 13:55:44,197 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:55:44,198 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-07 13:55:44,218 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45527 2023-08-07 13:55:44,219 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45527 2023-08-07 13:55:44,219 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45527 2023-08-07 13:55:44,220 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45527 2023-08-07 13:55:44,222 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45527 2023-08-07 13:55:44,223 INFO [Listener at localhost/41247] master.HMaster(444): hbase.rootdir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2, hbase.cluster.distributed=false 2023-08-07 13:55:44,248 INFO [Listener at localhost/41247] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-07 13:55:44,249 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,249 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,249 INFO [Listener at localhost/41247] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 13:55:44,249 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,249 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 13:55:44,249 INFO [Listener at localhost/41247] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 13:55:44,252 INFO [Listener at localhost/41247] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34709 2023-08-07 13:55:44,252 INFO [Listener at localhost/41247] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 13:55:44,257 DEBUG [Listener at localhost/41247] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 13:55:44,258 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:44,259 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:44,261 INFO [Listener at localhost/41247] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34709 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:44,268 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:347090x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:44,270 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:347090x0, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-07 13:55:44,270 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34709-0x101dfb6a1a10008 connected 2023-08-07 13:55:44,271 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:55:44,272 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-07 13:55:44,272 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34709 2023-08-07 13:55:44,286 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34709 2023-08-07 13:55:44,287 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34709 2023-08-07 13:55:44,288 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34709 2023-08-07 13:55:44,288 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34709 2023-08-07 13:55:44,305 INFO [Listener at localhost/41247] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-07 13:55:44,305 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,305 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,305 INFO [Listener at localhost/41247] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 13:55:44,305 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,306 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 13:55:44,306 INFO [Listener at localhost/41247] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 13:55:44,309 INFO [Listener at localhost/41247] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35577 2023-08-07 13:55:44,309 INFO [Listener at localhost/41247] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 13:55:44,311 DEBUG [Listener at localhost/41247] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 13:55:44,312 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:44,314 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:44,315 INFO [Listener at localhost/41247] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35577 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:44,320 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:355770x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:44,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:35577-0x101dfb6a1a10009 connected 2023-08-07 13:55:44,322 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-07 13:55:44,322 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:55:44,323 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-07 13:55:44,324 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35577 2023-08-07 13:55:44,324 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35577 2023-08-07 13:55:44,325 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35577 2023-08-07 13:55:44,327 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35577 2023-08-07 13:55:44,328 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35577 2023-08-07 13:55:44,343 INFO [Listener at localhost/41247] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-07 13:55:44,343 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,343 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,343 INFO [Listener at localhost/41247] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 13:55:44,343 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,343 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 13:55:44,343 INFO [Listener at localhost/41247] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 13:55:44,346 INFO [Listener at localhost/41247] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41049 2023-08-07 13:55:44,346 INFO [Listener at localhost/41247] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 13:55:44,349 DEBUG [Listener at localhost/41247] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 13:55:44,350 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:44,351 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:44,353 INFO [Listener at localhost/41247] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41049 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:44,358 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:410490x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:44,360 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:410490x0, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-07 13:55:44,360 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:410490x0, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:55:44,361 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:410490x0, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-07 13:55:44,363 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41049-0x101dfb6a1a1000a connected 2023-08-07 13:55:44,363 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41049 2023-08-07 13:55:44,366 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41049 2023-08-07 13:55:44,374 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41049 2023-08-07 13:55:44,378 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41049 2023-08-07 13:55:44,379 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41049 2023-08-07 13:55:44,394 INFO [Listener at localhost/41247] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-07 13:55:44,394 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,394 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,394 INFO [Listener at localhost/41247] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 13:55:44,394 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:44,394 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 13:55:44,394 INFO [Listener at localhost/41247] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 13:55:44,397 INFO [Listener at localhost/41247] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46589 2023-08-07 13:55:44,397 INFO [Listener at localhost/41247] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 13:55:44,399 DEBUG [Listener at localhost/41247] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 13:55:44,399 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:44,401 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:44,402 INFO [Listener at localhost/41247] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46589 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:44,415 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:465890x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:44,417 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:465890x0, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-07 13:55:44,418 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46589-0x101dfb6a1a1000b connected 2023-08-07 13:55:44,418 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:55:44,419 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-07 13:55:44,427 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46589 2023-08-07 13:55:44,427 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46589 2023-08-07 13:55:44,427 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46589 2023-08-07 13:55:44,435 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46589 2023-08-07 13:55:44,437 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46589 2023-08-07 13:55:44,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,45527,1691416544159 2023-08-07 13:55:44,446 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-07 13:55:44,447 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,45527,1691416544159 2023-08-07 13:55:44,448 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:41049-0x101dfb6a1a1000a, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-07 13:55:44,448 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-07 13:55:44,449 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-07 13:55:44,449 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-07 13:55:44,449 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-07 13:55:44,450 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:55:44,454 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-07 13:55:44,455 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-07 13:55:44,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,45527,1691416544159 from backup master directory 2023-08-07 13:55:44,457 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,45527,1691416544159 2023-08-07 13:55:44,458 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-07 13:55:44,458 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 13:55:44,458 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,45527,1691416544159 2023-08-07 13:55:44,502 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/hbase.id with ID: bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:55:44,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:44,525 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:55:44,550 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x17cec3ba to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:44,555 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@16911353, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:44,555 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 13:55:44,556 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-07 13:55:44,556 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:55:44,557 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:55:44,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store-tmp 2023-08-07 13:55:44,592 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:44,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 13:55:44,593 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:44,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:44,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 13:55:44,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:44,593 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:44,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 13:55:44,594 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,45527,1691416544159 2023-08-07 13:55:44,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:44,599 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45527%2C1691416544159.jenkins-hbase4.apache.org%2C45527%2C1691416544159.regiongroup-0, suffix=, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,45527,1691416544159, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/oldWALs, maxLogs=10 2023-08-07 13:55:44,624 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:55:44,624 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK] 2023-08-07 13:55:44,624 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK] 2023-08-07 13:55:44,634 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,45527,1691416544159/jenkins-hbase4.apache.org%2C45527%2C1691416544159.jenkins-hbase4.apache.org%2C45527%2C1691416544159.regiongroup-0.1691416544600 2023-08-07 13:55:44,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK], DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK], DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK]] 2023-08-07 13:55:44,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:44,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:44,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:44,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:44,639 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:44,642 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-07 13:55:44,642 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-07 13:55:44,644 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:44,646 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:44,647 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:44,652 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:44,655 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 13:55:44,656 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69532045, jitterRate=0.03610821068286896}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 13:55:44,656 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 13:55:44,656 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-07 13:55:44,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-07 13:55:44,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-07 13:55:44,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 13:55:44,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 13:55:44,660 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 1 msec 2023-08-07 13:55:44,660 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-07 13:55:44,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-07 13:55:44,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-07 13:55:44,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-07 13:55:44,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-07 13:55:44,677 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-08-07 13:55:44,677 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-07 13:55:44,678 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-08-07 13:55:44,680 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:55:44,682 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-08-07 13:55:44,683 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-08-07 13:55:44,684 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-08-07 13:55:44,685 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:41049-0x101dfb6a1a1000a, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-07 13:55:44,685 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-07 13:55:44,686 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-07 13:55:44,685 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-07 13:55:44,685 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-07 13:55:44,686 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:55:44,686 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,45527,1691416544159, sessionid=0x101dfb6a1a10007, setting cluster-up flag (Was=false) 2023-08-07 13:55:44,702 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:55:44,711 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-08-07 13:55:44,713 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45527,1691416544159 2023-08-07 13:55:44,716 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:55:44,724 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-08-07 13:55:44,725 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45527,1691416544159 2023-08-07 13:55:44,726 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.hbase-snapshot/.tmp 2023-08-07 13:55:44,729 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-07 13:55:44,730 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:55:44,730 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:55:44,730 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:55:44,730 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:55:44,730 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-07 13:55:44,730 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,730 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 13:55:44,730 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,732 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691416574732 2023-08-07 13:55:44,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-07 13:55:44,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-07 13:55:44,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-07 13:55:44,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-07 13:55:44,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-07 13:55:44,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-07 13:55:44,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-07 13:55:44,734 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-07 13:55:44,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-07 13:55:44,734 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-07 13:55:44,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-07 13:55:44,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-07 13:55:44,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-07 13:55:44,735 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416544735,5,FailOnTimeoutGroup] 2023-08-07 13:55:44,736 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416544735,5,FailOnTimeoutGroup] 2023-08-07 13:55:44,736 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,736 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-07 13:55:44,736 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-07 13:55:44,736 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,736 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,739 INFO [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(951): ClusterId : bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:55:44,739 INFO [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(951): ClusterId : bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:55:44,745 DEBUG [RS:0;jenkins-hbase4:34709] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 13:55:44,745 DEBUG [RS:1;jenkins-hbase4:35577] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 13:55:44,748 INFO [RS:2;jenkins-hbase4:41049] regionserver.HRegionServer(951): ClusterId : bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:55:44,749 DEBUG [RS:2;jenkins-hbase4:41049] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 13:55:44,752 INFO [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(951): ClusterId : bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:55:44,754 DEBUG [RS:3;jenkins-hbase4:46589] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 13:55:44,754 DEBUG [RS:1;jenkins-hbase4:35577] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 13:55:44,754 DEBUG [RS:1;jenkins-hbase4:35577] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 13:55:44,755 DEBUG [RS:0;jenkins-hbase4:34709] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 13:55:44,755 DEBUG [RS:0;jenkins-hbase4:34709] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 13:55:44,767 DEBUG [RS:2;jenkins-hbase4:41049] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 13:55:44,768 DEBUG [RS:2;jenkins-hbase4:41049] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 13:55:44,768 DEBUG [RS:0;jenkins-hbase4:34709] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 13:55:44,768 DEBUG [RS:3;jenkins-hbase4:46589] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 13:55:44,769 DEBUG [RS:3;jenkins-hbase4:46589] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 13:55:44,768 DEBUG [RS:1;jenkins-hbase4:35577] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 13:55:44,771 DEBUG [RS:0;jenkins-hbase4:34709] zookeeper.ReadOnlyZKClient(139): Connect 0x473cb62f to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:44,771 DEBUG [RS:1;jenkins-hbase4:35577] zookeeper.ReadOnlyZKClient(139): Connect 0x3970f1d4 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:44,779 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-07 13:55:44,780 DEBUG [RS:3;jenkins-hbase4:46589] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 13:55:44,780 DEBUG [RS:2;jenkins-hbase4:41049] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 13:55:44,782 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-07 13:55:44,782 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2 2023-08-07 13:55:44,782 DEBUG [RS:0;jenkins-hbase4:34709] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@49fb40b8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:44,783 DEBUG [RS:2;jenkins-hbase4:41049] zookeeper.ReadOnlyZKClient(139): Connect 0x013efa24 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:44,783 DEBUG [RS:3;jenkins-hbase4:46589] zookeeper.ReadOnlyZKClient(139): Connect 0x4e7ae206 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:44,784 DEBUG [RS:1;jenkins-hbase4:35577] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@33889aa4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:44,784 DEBUG [RS:0;jenkins-hbase4:34709] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@8c64a83, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 13:55:44,784 DEBUG [RS:1;jenkins-hbase4:35577] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@768a12b1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 13:55:44,795 DEBUG [RS:2;jenkins-hbase4:41049] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5017edb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:44,795 DEBUG [RS:2;jenkins-hbase4:41049] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4003f01f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 13:55:44,798 DEBUG [RS:3;jenkins-hbase4:46589] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4a457f11, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:44,798 DEBUG [RS:3;jenkins-hbase4:46589] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@f47e83d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 13:55:44,798 DEBUG [RS:0;jenkins-hbase4:34709] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:34709 2023-08-07 13:55:44,798 INFO [RS:0;jenkins-hbase4:34709] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 13:55:44,798 INFO [RS:0;jenkins-hbase4:34709] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 13:55:44,798 DEBUG [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 13:55:44,800 DEBUG [RS:1;jenkins-hbase4:35577] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:35577 2023-08-07 13:55:44,800 INFO [RS:1;jenkins-hbase4:35577] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 13:55:44,800 INFO [RS:1;jenkins-hbase4:35577] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 13:55:44,800 DEBUG [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 13:55:44,812 INFO [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45527,1691416544159 with isa=jenkins-hbase4.apache.org/172.31.14.131:34709, startcode=1691416544248 2023-08-07 13:55:44,812 INFO [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45527,1691416544159 with isa=jenkins-hbase4.apache.org/172.31.14.131:35577, startcode=1691416544304 2023-08-07 13:55:44,812 DEBUG [RS:0;jenkins-hbase4:34709] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 13:55:44,813 DEBUG [RS:2;jenkins-hbase4:41049] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:41049 2023-08-07 13:55:44,812 DEBUG [RS:1;jenkins-hbase4:35577] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 13:55:44,814 INFO [RS:2;jenkins-hbase4:41049] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 13:55:44,814 INFO [RS:2;jenkins-hbase4:41049] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 13:55:44,814 DEBUG [RS:2;jenkins-hbase4:41049] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 13:55:44,815 INFO [RS:2;jenkins-hbase4:41049] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45527,1691416544159 with isa=jenkins-hbase4.apache.org/172.31.14.131:41049, startcode=1691416544342 2023-08-07 13:55:44,815 DEBUG [RS:2;jenkins-hbase4:41049] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 13:55:44,821 DEBUG [RS:3;jenkins-hbase4:46589] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase4:46589 2023-08-07 13:55:44,822 INFO [RS:3;jenkins-hbase4:46589] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 13:55:44,823 INFO [RS:3;jenkins-hbase4:46589] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 13:55:44,823 DEBUG [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 13:55:44,825 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40353, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 13:55:44,825 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43811, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 13:55:44,825 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57675, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 13:55:44,827 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45527] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,35577,1691416544304 2023-08-07 13:55:44,827 INFO [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45527,1691416544159 with isa=jenkins-hbase4.apache.org/172.31.14.131:46589, startcode=1691416544393 2023-08-07 13:55:44,827 DEBUG [RS:3;jenkins-hbase4:46589] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 13:55:44,827 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45527] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:44,828 DEBUG [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2 2023-08-07 13:55:44,828 DEBUG [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45993 2023-08-07 13:55:44,828 DEBUG [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 13:55:44,828 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45527] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41049,1691416544342 2023-08-07 13:55:44,828 DEBUG [RS:2;jenkins-hbase4:41049] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2 2023-08-07 13:55:44,828 DEBUG [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2 2023-08-07 13:55:44,828 DEBUG [RS:2;jenkins-hbase4:41049] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45993 2023-08-07 13:55:44,829 DEBUG [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45993 2023-08-07 13:55:44,829 DEBUG [RS:2;jenkins-hbase4:41049] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 13:55:44,829 DEBUG [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 13:55:44,832 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:44,832 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45387, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 13:55:44,833 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45527] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:44,834 DEBUG [RS:1;jenkins-hbase4:35577] zookeeper.ZKUtil(162): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,35577,1691416544304 2023-08-07 13:55:44,834 WARN [RS:1;jenkins-hbase4:35577] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 13:55:44,834 INFO [RS:1;jenkins-hbase4:35577] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:55:44,834 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,35577,1691416544304] 2023-08-07 13:55:44,834 INFO [RS:1;jenkins-hbase4:35577] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:55:44,834 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34709,1691416544248] 2023-08-07 13:55:44,834 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41049,1691416544342] 2023-08-07 13:55:44,834 DEBUG [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,35577,1691416544304 2023-08-07 13:55:44,834 DEBUG [RS:2;jenkins-hbase4:41049] zookeeper.ZKUtil(162): regionserver:41049-0x101dfb6a1a1000a, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,41049,1691416544342 2023-08-07 13:55:44,834 DEBUG [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2 2023-08-07 13:55:44,834 WARN [RS:2;jenkins-hbase4:41049] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 13:55:44,835 INFO [RS:2;jenkins-hbase4:41049] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:55:44,835 DEBUG [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45993 2023-08-07 13:55:44,835 INFO [RS:2;jenkins-hbase4:41049] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:55:44,835 DEBUG [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 13:55:44,835 DEBUG [RS:2;jenkins-hbase4:41049] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,41049,1691416544342 2023-08-07 13:55:44,835 DEBUG [RS:0;jenkins-hbase4:34709] zookeeper.ZKUtil(162): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:44,835 WARN [RS:0;jenkins-hbase4:34709] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 13:55:44,835 INFO [RS:0;jenkins-hbase4:34709] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:55:44,835 INFO [RS:0;jenkins-hbase4:34709] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:55:44,835 DEBUG [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:44,836 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:44,837 DEBUG [RS:3;jenkins-hbase4:46589] zookeeper.ZKUtil(162): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:44,837 WARN [RS:3;jenkins-hbase4:46589] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 13:55:44,837 INFO [RS:3;jenkins-hbase4:46589] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:55:44,837 INFO [RS:3;jenkins-hbase4:46589] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:55:44,837 DEBUG [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:44,845 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46589,1691416544393] 2023-08-07 13:55:44,850 DEBUG [RS:0;jenkins-hbase4:34709] zookeeper.ZKUtil(162): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,35577,1691416544304 2023-08-07 13:55:44,851 DEBUG [RS:2;jenkins-hbase4:41049] zookeeper.ZKUtil(162): regionserver:41049-0x101dfb6a1a1000a, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,35577,1691416544304 2023-08-07 13:55:44,851 DEBUG [RS:0;jenkins-hbase4:34709] zookeeper.ZKUtil(162): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:44,853 DEBUG [RS:3;jenkins-hbase4:46589] zookeeper.ZKUtil(162): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,35577,1691416544304 2023-08-07 13:55:44,853 DEBUG [RS:0;jenkins-hbase4:34709] zookeeper.ZKUtil(162): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,41049,1691416544342 2023-08-07 13:55:44,854 DEBUG [RS:2;jenkins-hbase4:41049] zookeeper.ZKUtil(162): regionserver:41049-0x101dfb6a1a1000a, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:44,854 DEBUG [RS:0;jenkins-hbase4:34709] zookeeper.ZKUtil(162): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:44,854 DEBUG [RS:3;jenkins-hbase4:46589] zookeeper.ZKUtil(162): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:44,854 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:44,856 DEBUG [RS:1;jenkins-hbase4:35577] zookeeper.ZKUtil(162): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,35577,1691416544304 2023-08-07 13:55:44,856 DEBUG [RS:2;jenkins-hbase4:41049] zookeeper.ZKUtil(162): regionserver:41049-0x101dfb6a1a1000a, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,41049,1691416544342 2023-08-07 13:55:44,856 DEBUG [RS:3;jenkins-hbase4:46589] zookeeper.ZKUtil(162): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,41049,1691416544342 2023-08-07 13:55:44,856 DEBUG [RS:1;jenkins-hbase4:35577] zookeeper.ZKUtil(162): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:44,856 DEBUG [RS:2;jenkins-hbase4:41049] zookeeper.ZKUtil(162): regionserver:41049-0x101dfb6a1a1000a, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:44,857 DEBUG [RS:3;jenkins-hbase4:46589] zookeeper.ZKUtil(162): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:44,857 DEBUG [RS:1;jenkins-hbase4:35577] zookeeper.ZKUtil(162): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,41049,1691416544342 2023-08-07 13:55:44,857 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 13:55:44,858 DEBUG [RS:0;jenkins-hbase4:34709] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-07 13:55:44,858 INFO [RS:0;jenkins-hbase4:34709] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 13:55:44,858 DEBUG [RS:1;jenkins-hbase4:35577] zookeeper.ZKUtil(162): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:44,859 DEBUG [RS:2;jenkins-hbase4:41049] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-07 13:55:44,860 DEBUG [RS:3;jenkins-hbase4:46589] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-07 13:55:44,860 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info 2023-08-07 13:55:44,861 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 13:55:44,861 INFO [RS:3;jenkins-hbase4:46589] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 13:55:44,861 INFO [RS:0;jenkins-hbase4:34709] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 13:55:44,863 INFO [RS:0;jenkins-hbase4:34709] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 13:55:44,863 INFO [RS:0;jenkins-hbase4:34709] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,863 INFO [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 13:55:44,863 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:44,863 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 13:55:44,865 DEBUG [RS:1;jenkins-hbase4:35577] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-07 13:55:44,866 INFO [RS:0;jenkins-hbase4:34709] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,866 DEBUG [RS:0;jenkins-hbase4:34709] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,866 DEBUG [RS:0;jenkins-hbase4:34709] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,866 DEBUG [RS:0;jenkins-hbase4:34709] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,866 DEBUG [RS:0;jenkins-hbase4:34709] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,866 DEBUG [RS:0;jenkins-hbase4:34709] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,866 DEBUG [RS:0;jenkins-hbase4:34709] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 13:55:44,866 DEBUG [RS:0;jenkins-hbase4:34709] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,866 DEBUG [RS:0;jenkins-hbase4:34709] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,866 DEBUG [RS:0;jenkins-hbase4:34709] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,866 DEBUG [RS:0;jenkins-hbase4:34709] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,867 INFO [RS:1;jenkins-hbase4:35577] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 13:55:44,867 INFO [RS:2;jenkins-hbase4:41049] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 13:55:44,868 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/rep_barrier 2023-08-07 13:55:44,868 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 13:55:44,869 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:44,869 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 13:55:44,871 INFO [RS:3;jenkins-hbase4:46589] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 13:55:44,871 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/table 2023-08-07 13:55:44,875 INFO [RS:1;jenkins-hbase4:35577] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 13:55:44,876 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 13:55:44,876 INFO [RS:0;jenkins-hbase4:34709] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,876 INFO [RS:0;jenkins-hbase4:34709] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,876 INFO [RS:0;jenkins-hbase4:34709] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,877 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:44,877 INFO [RS:3;jenkins-hbase4:46589] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 13:55:44,887 INFO [RS:3;jenkins-hbase4:46589] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,887 INFO [RS:1;jenkins-hbase4:35577] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 13:55:44,888 INFO [RS:2;jenkins-hbase4:41049] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 13:55:44,888 INFO [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 13:55:44,890 INFO [RS:2;jenkins-hbase4:41049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 13:55:44,890 INFO [RS:2;jenkins-hbase4:41049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,888 INFO [RS:1;jenkins-hbase4:35577] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,891 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740 2023-08-07 13:55:44,891 INFO [RS:2;jenkins-hbase4:41049] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 13:55:44,891 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740 2023-08-07 13:55:44,892 INFO [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 13:55:44,895 INFO [RS:3;jenkins-hbase4:46589] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,895 DEBUG [RS:3;jenkins-hbase4:46589] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,895 DEBUG [RS:3;jenkins-hbase4:46589] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,895 DEBUG [RS:3;jenkins-hbase4:46589] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,895 INFO [RS:1;jenkins-hbase4:35577] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,895 DEBUG [RS:3;jenkins-hbase4:46589] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,899 DEBUG [RS:3;jenkins-hbase4:46589] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,899 DEBUG [RS:3;jenkins-hbase4:46589] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 13:55:44,899 DEBUG [RS:3;jenkins-hbase4:46589] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,900 DEBUG [RS:3;jenkins-hbase4:46589] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,900 DEBUG [RS:3;jenkins-hbase4:46589] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,900 DEBUG [RS:3;jenkins-hbase4:46589] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,913 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-07 13:55:44,914 DEBUG [RS:1;jenkins-hbase4:35577] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,915 DEBUG [RS:1;jenkins-hbase4:35577] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,915 DEBUG [RS:1;jenkins-hbase4:35577] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,915 DEBUG [RS:1;jenkins-hbase4:35577] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,915 DEBUG [RS:1;jenkins-hbase4:35577] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,915 DEBUG [RS:1;jenkins-hbase4:35577] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 13:55:44,915 DEBUG [RS:1;jenkins-hbase4:35577] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,915 DEBUG [RS:1;jenkins-hbase4:35577] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,915 DEBUG [RS:1;jenkins-hbase4:35577] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,915 DEBUG [RS:1;jenkins-hbase4:35577] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,916 INFO [RS:3;jenkins-hbase4:46589] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,916 INFO [RS:3;jenkins-hbase4:46589] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,916 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 13:55:44,916 INFO [RS:3;jenkins-hbase4:46589] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,919 INFO [RS:0;jenkins-hbase4:34709] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 13:55:44,919 INFO [RS:0;jenkins-hbase4:34709] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34709,1691416544248-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,922 INFO [RS:1;jenkins-hbase4:35577] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,923 INFO [RS:1;jenkins-hbase4:35577] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,923 INFO [RS:1;jenkins-hbase4:35577] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,924 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 13:55:44,925 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60801798, jitterRate=-0.09398260712623596}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-07 13:55:44,925 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 13:55:44,925 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 13:55:44,926 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 13:55:44,926 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 13:55:44,926 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 13:55:44,926 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 13:55:44,931 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 13:55:44,931 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 13:55:44,931 INFO [RS:2;jenkins-hbase4:41049] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,932 DEBUG [RS:2;jenkins-hbase4:41049] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,932 DEBUG [RS:2;jenkins-hbase4:41049] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,932 DEBUG [RS:2;jenkins-hbase4:41049] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,932 DEBUG [RS:2;jenkins-hbase4:41049] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,932 DEBUG [RS:2;jenkins-hbase4:41049] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,932 DEBUG [RS:2;jenkins-hbase4:41049] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 13:55:44,932 DEBUG [RS:2;jenkins-hbase4:41049] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,932 DEBUG [RS:2;jenkins-hbase4:41049] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,932 DEBUG [RS:2;jenkins-hbase4:41049] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,933 DEBUG [RS:2;jenkins-hbase4:41049] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:44,938 INFO [RS:2;jenkins-hbase4:41049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,938 INFO [RS:2;jenkins-hbase4:41049] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,938 INFO [RS:2;jenkins-hbase4:41049] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,949 INFO [RS:3;jenkins-hbase4:46589] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 13:55:44,950 INFO [RS:3;jenkins-hbase4:46589] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46589,1691416544393-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,958 INFO [RS:2;jenkins-hbase4:41049] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 13:55:44,958 INFO [RS:2;jenkins-hbase4:41049] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41049,1691416544342-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,962 INFO [RS:1;jenkins-hbase4:35577] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 13:55:44,963 INFO [RS:1;jenkins-hbase4:35577] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35577,1691416544304-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:44,964 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-07 13:55:44,964 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-07 13:55:44,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-07 13:55:44,972 INFO [RS:0;jenkins-hbase4:34709] regionserver.Replication(203): jenkins-hbase4.apache.org,34709,1691416544248 started 2023-08-07 13:55:44,972 INFO [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34709,1691416544248, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34709, sessionid=0x101dfb6a1a10008 2023-08-07 13:55:44,973 INFO [RS:3;jenkins-hbase4:46589] regionserver.Replication(203): jenkins-hbase4.apache.org,46589,1691416544393 started 2023-08-07 13:55:44,973 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-07 13:55:44,973 INFO [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46589,1691416544393, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46589, sessionid=0x101dfb6a1a1000b 2023-08-07 13:55:44,973 DEBUG [RS:0;jenkins-hbase4:34709] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 13:55:44,973 DEBUG [RS:3;jenkins-hbase4:46589] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 13:55:44,973 DEBUG [RS:3;jenkins-hbase4:46589] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:44,973 DEBUG [RS:0;jenkins-hbase4:34709] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:44,973 DEBUG [RS:3;jenkins-hbase4:46589] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46589,1691416544393' 2023-08-07 13:55:44,973 DEBUG [RS:0;jenkins-hbase4:34709] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34709,1691416544248' 2023-08-07 13:55:44,973 DEBUG [RS:3;jenkins-hbase4:46589] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-07 13:55:44,974 DEBUG [RS:0;jenkins-hbase4:34709] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-07 13:55:44,975 DEBUG [RS:3;jenkins-hbase4:46589] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-07 13:55:44,975 DEBUG [RS:0;jenkins-hbase4:34709] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-07 13:55:44,977 DEBUG [RS:3;jenkins-hbase4:46589] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 13:55:44,977 DEBUG [RS:3;jenkins-hbase4:46589] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 13:55:44,977 DEBUG [RS:3;jenkins-hbase4:46589] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:44,977 DEBUG [RS:3;jenkins-hbase4:46589] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46589,1691416544393' 2023-08-07 13:55:44,977 DEBUG [RS:3;jenkins-hbase4:46589] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-07 13:55:44,977 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-07 13:55:44,977 DEBUG [RS:0;jenkins-hbase4:34709] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 13:55:44,977 DEBUG [RS:0;jenkins-hbase4:34709] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 13:55:44,977 DEBUG [RS:0;jenkins-hbase4:34709] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:44,977 DEBUG [RS:0;jenkins-hbase4:34709] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34709,1691416544248' 2023-08-07 13:55:44,977 DEBUG [RS:3;jenkins-hbase4:46589] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-07 13:55:44,977 DEBUG [RS:0;jenkins-hbase4:34709] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-07 13:55:44,978 DEBUG [RS:3;jenkins-hbase4:46589] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 13:55:44,978 INFO [RS:3;jenkins-hbase4:46589] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 13:55:44,978 INFO [RS:3;jenkins-hbase4:46589] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 13:55:44,978 DEBUG [RS:0;jenkins-hbase4:34709] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-07 13:55:44,980 DEBUG [RS:0;jenkins-hbase4:34709] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 13:55:44,981 INFO [RS:0;jenkins-hbase4:34709] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 13:55:44,981 INFO [RS:0;jenkins-hbase4:34709] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 13:55:44,982 INFO [RS:2;jenkins-hbase4:41049] regionserver.Replication(203): jenkins-hbase4.apache.org,41049,1691416544342 started 2023-08-07 13:55:44,983 INFO [RS:2;jenkins-hbase4:41049] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,41049,1691416544342, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41049, sessionid=0x101dfb6a1a1000a 2023-08-07 13:55:44,983 DEBUG [RS:2;jenkins-hbase4:41049] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 13:55:44,983 DEBUG [RS:2;jenkins-hbase4:41049] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41049,1691416544342 2023-08-07 13:55:44,983 DEBUG [RS:2;jenkins-hbase4:41049] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41049,1691416544342' 2023-08-07 13:55:44,983 DEBUG [RS:2;jenkins-hbase4:41049] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-07 13:55:44,983 DEBUG [RS:2;jenkins-hbase4:41049] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-07 13:55:44,984 DEBUG [RS:2;jenkins-hbase4:41049] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 13:55:44,984 DEBUG [RS:2;jenkins-hbase4:41049] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 13:55:44,984 DEBUG [RS:2;jenkins-hbase4:41049] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41049,1691416544342 2023-08-07 13:55:44,984 DEBUG [RS:2;jenkins-hbase4:41049] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41049,1691416544342' 2023-08-07 13:55:44,984 DEBUG [RS:2;jenkins-hbase4:41049] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-07 13:55:44,984 DEBUG [RS:2;jenkins-hbase4:41049] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-07 13:55:44,985 DEBUG [RS:2;jenkins-hbase4:41049] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 13:55:44,985 INFO [RS:2;jenkins-hbase4:41049] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 13:55:44,985 INFO [RS:2;jenkins-hbase4:41049] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 13:55:44,985 INFO [RS:1;jenkins-hbase4:35577] regionserver.Replication(203): jenkins-hbase4.apache.org,35577,1691416544304 started 2023-08-07 13:55:44,985 INFO [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,35577,1691416544304, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:35577, sessionid=0x101dfb6a1a10009 2023-08-07 13:55:44,985 DEBUG [RS:1;jenkins-hbase4:35577] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 13:55:44,985 DEBUG [RS:1;jenkins-hbase4:35577] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,35577,1691416544304 2023-08-07 13:55:44,985 DEBUG [RS:1;jenkins-hbase4:35577] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35577,1691416544304' 2023-08-07 13:55:44,985 DEBUG [RS:1;jenkins-hbase4:35577] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-07 13:55:44,986 DEBUG [RS:1;jenkins-hbase4:35577] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-07 13:55:44,986 DEBUG [RS:1;jenkins-hbase4:35577] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 13:55:44,986 DEBUG [RS:1;jenkins-hbase4:35577] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 13:55:44,987 DEBUG [RS:1;jenkins-hbase4:35577] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,35577,1691416544304 2023-08-07 13:55:44,987 DEBUG [RS:1;jenkins-hbase4:35577] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35577,1691416544304' 2023-08-07 13:55:44,987 DEBUG [RS:1;jenkins-hbase4:35577] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-07 13:55:44,987 DEBUG [RS:1;jenkins-hbase4:35577] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-07 13:55:44,988 DEBUG [RS:1;jenkins-hbase4:35577] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 13:55:44,988 INFO [RS:1;jenkins-hbase4:35577] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 13:55:44,988 INFO [RS:1;jenkins-hbase4:35577] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 13:55:45,079 INFO [RS:3;jenkins-hbase4:46589] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:45,081 INFO [RS:0;jenkins-hbase4:34709] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:45,082 INFO [RS:3;jenkins-hbase4:46589] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46589%2C1691416544393.jenkins-hbase4.apache.org%2C46589%2C1691416544393.regiongroup-0, suffix=, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46589,1691416544393, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs, maxLogs=10 2023-08-07 13:55:45,084 INFO [RS:0;jenkins-hbase4:34709] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34709%2C1691416544248.jenkins-hbase4.apache.org%2C34709%2C1691416544248.regiongroup-0, suffix=, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,34709,1691416544248, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs, maxLogs=10 2023-08-07 13:55:45,085 INFO [RS:2;jenkins-hbase4:41049] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:45,088 INFO [RS:1;jenkins-hbase4:35577] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:45,092 INFO [RS:2;jenkins-hbase4:41049] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41049%2C1691416544342.jenkins-hbase4.apache.org%2C41049%2C1691416544342.regiongroup-0, suffix=, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,41049,1691416544342, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs, maxLogs=10 2023-08-07 13:55:45,093 INFO [RS:1;jenkins-hbase4:35577] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35577%2C1691416544304.jenkins-hbase4.apache.org%2C35577%2C1691416544304.regiongroup-0, suffix=, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,35577,1691416544304, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs, maxLogs=10 2023-08-07 13:55:45,117 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK] 2023-08-07 13:55:45,117 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK] 2023-08-07 13:55:45,117 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:55:45,127 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK] 2023-08-07 13:55:45,130 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK] 2023-08-07 13:55:45,130 DEBUG [jenkins-hbase4:45527] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=4, allServersCount=4 2023-08-07 13:55:45,128 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:55:45,131 DEBUG [jenkins-hbase4:45527] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 13:55:45,134 DEBUG [jenkins-hbase4:45527] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 13:55:45,134 DEBUG [jenkins-hbase4:45527] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 13:55:45,134 DEBUG [jenkins-hbase4:45527] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 13:55:45,134 DEBUG [jenkins-hbase4:45527] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-08-07 13:55:45,134 DEBUG [jenkins-hbase4:45527] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 13:55:45,137 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46589,1691416544393, state=OPENING 2023-08-07 13:55:45,146 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK] 2023-08-07 13:55:45,147 DEBUG [PEWorker-2] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-07 13:55:45,149 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:55:45,150 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46589,1691416544393}] 2023-08-07 13:55:45,151 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK] 2023-08-07 13:55:45,151 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK] 2023-08-07 13:55:45,150 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-07 13:55:45,154 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:55:45,154 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:55:45,155 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK] 2023-08-07 13:55:45,155 INFO [RS:3;jenkins-hbase4:46589] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46589,1691416544393/jenkins-hbase4.apache.org%2C46589%2C1691416544393.jenkins-hbase4.apache.org%2C46589%2C1691416544393.regiongroup-0.1691416545083 2023-08-07 13:55:45,156 INFO [RS:0;jenkins-hbase4:34709] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,34709,1691416544248/jenkins-hbase4.apache.org%2C34709%2C1691416544248.jenkins-hbase4.apache.org%2C34709%2C1691416544248.regiongroup-0.1691416545085 2023-08-07 13:55:45,168 DEBUG [RS:3;jenkins-hbase4:46589] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK], DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK], DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK]] 2023-08-07 13:55:45,169 DEBUG [RS:0;jenkins-hbase4:34709] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK], DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK], DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK]] 2023-08-07 13:55:45,171 INFO [RS:2;jenkins-hbase4:41049] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,41049,1691416544342/jenkins-hbase4.apache.org%2C41049%2C1691416544342.jenkins-hbase4.apache.org%2C41049%2C1691416544342.regiongroup-0.1691416545093 2023-08-07 13:55:45,184 INFO [RS:1;jenkins-hbase4:35577] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,35577,1691416544304/jenkins-hbase4.apache.org%2C35577%2C1691416544304.jenkins-hbase4.apache.org%2C35577%2C1691416544304.regiongroup-0.1691416545094 2023-08-07 13:55:45,186 DEBUG [RS:2;jenkins-hbase4:41049] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK], DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK], DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK]] 2023-08-07 13:55:45,186 DEBUG [RS:1;jenkins-hbase4:35577] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK], DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK], DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK]] 2023-08-07 13:55:45,325 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:45,326 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 13:55:45,328 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59108, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 13:55:45,336 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-07 13:55:45,336 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:55:45,337 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:55:45,337 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:45,339 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46589%2C1691416544393.meta, suffix=.meta, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46589,1691416544393, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs, maxLogs=10 2023-08-07 13:55:45,358 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK] 2023-08-07 13:55:45,360 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK] 2023-08-07 13:55:45,360 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:55:45,379 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46589,1691416544393/jenkins-hbase4.apache.org%2C46589%2C1691416544393.meta.1691416545340.meta 2023-08-07 13:55:45,379 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK], DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK], DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK]] 2023-08-07 13:55:45,380 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:45,380 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-07 13:55:45,380 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-07 13:55:45,381 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-07 13:55:45,381 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-07 13:55:45,381 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:45,381 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-07 13:55:45,381 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-07 13:55:45,386 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 13:55:45,388 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info 2023-08-07 13:55:45,388 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info 2023-08-07 13:55:45,389 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 13:55:45,390 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:45,390 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 13:55:45,392 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/rep_barrier 2023-08-07 13:55:45,392 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/rep_barrier 2023-08-07 13:55:45,392 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 13:55:45,393 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:45,393 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 13:55:45,395 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/table 2023-08-07 13:55:45,395 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/table 2023-08-07 13:55:45,395 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 13:55:45,396 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:45,398 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740 2023-08-07 13:55:45,401 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740 2023-08-07 13:55:45,405 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-07 13:55:45,408 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 13:55:45,409 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64227018, jitterRate=-0.04294285178184509}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-07 13:55:45,409 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 13:55:45,413 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1691416545325 2023-08-07 13:55:45,422 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-07 13:55:45,423 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-07 13:55:45,424 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46589,1691416544393, state=OPEN 2023-08-07 13:55:45,426 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-07 13:55:45,426 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-07 13:55:45,431 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-07 13:55:45,431 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46589,1691416544393 in 277 msec 2023-08-07 13:55:45,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-07 13:55:45,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 467 msec 2023-08-07 13:55:45,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 710 msec 2023-08-07 13:55:45,439 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691416545439, completionTime=-1 2023-08-07 13:55:45,439 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=4; waited=0ms, expected min=4 server(s), max=4 server(s), master is running 2023-08-07 13:55:45,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 13:55:45,448 DEBUG [hconnection-0x269f83ed-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:45,450 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59120, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:55:45,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=4 2023-08-07 13:55:45,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691416605453 2023-08-07 13:55:45,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691416665453 2023-08-07 13:55:45,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 13 msec 2023-08-07 13:55:45,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45527,1691416544159-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:45,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45527,1691416544159-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:45,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45527,1691416544159-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:45,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:45527, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:45,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:45,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-07 13:55:45,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-07 13:55:45,473 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-07 13:55:45,474 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-07 13:55:45,476 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 13:55:45,478 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 13:55:45,484 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:45,485 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147 empty. 2023-08-07 13:55:45,486 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:45,486 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-07 13:55:45,542 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-07 13:55:45,543 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3bc256268a4ab1eec2fd5f34b09fb147, NAME => 'hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp 2023-08-07 13:55:45,563 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:45,564 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 3bc256268a4ab1eec2fd5f34b09fb147, disabling compactions & flushes 2023-08-07 13:55:45,564 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:45,564 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:45,564 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. after waiting 0 ms 2023-08-07 13:55:45,564 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:45,564 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:45,564 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 3bc256268a4ab1eec2fd5f34b09fb147: 2023-08-07 13:55:45,569 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 13:55:45,571 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691416545571"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691416545571"}]},"ts":"1691416545571"} 2023-08-07 13:55:45,580 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 13:55:45,582 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 13:55:45,583 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691416545582"}]},"ts":"1691416545582"} 2023-08-07 13:55:45,586 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-07 13:55:45,592 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 13:55:45,593 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 13:55:45,594 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 13:55:45,594 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 13:55:45,594 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-08-07 13:55:45,594 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 13:55:45,594 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=3bc256268a4ab1eec2fd5f34b09fb147, ASSIGN}] 2023-08-07 13:55:45,598 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=3bc256268a4ab1eec2fd5f34b09fb147, ASSIGN 2023-08-07 13:55:45,600 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=3bc256268a4ab1eec2fd5f34b09fb147, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46589,1691416544393; forceNewPlan=false, retain=false 2023-08-07 13:55:45,750 INFO [jenkins-hbase4:45527] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 13:55:45,752 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=3bc256268a4ab1eec2fd5f34b09fb147, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:45,753 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691416545752"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416545752"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416545752"}]},"ts":"1691416545752"} 2023-08-07 13:55:45,760 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 3bc256268a4ab1eec2fd5f34b09fb147, server=jenkins-hbase4.apache.org,46589,1691416544393}] 2023-08-07 13:55:45,924 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:45,925 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:45,928 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46589%2C1691416544393.jenkins-hbase4.apache.org%2C46589%2C1691416544393.regiongroup-1, suffix=, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46589,1691416544393, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs, maxLogs=10 2023-08-07 13:55:45,949 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK] 2023-08-07 13:55:45,951 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:55:45,951 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK] 2023-08-07 13:55:45,960 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46589,1691416544393/jenkins-hbase4.apache.org%2C46589%2C1691416544393.jenkins-hbase4.apache.org%2C46589%2C1691416544393.regiongroup-1.1691416545928 2023-08-07 13:55:45,962 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK], DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK], DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK]] 2023-08-07 13:55:45,963 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3bc256268a4ab1eec2fd5f34b09fb147, NAME => 'hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:45,963 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:45,963 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:45,964 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:45,964 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:45,967 INFO [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:45,969 DEBUG [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/info 2023-08-07 13:55:45,969 DEBUG [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/info 2023-08-07 13:55:45,971 INFO [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bc256268a4ab1eec2fd5f34b09fb147 columnFamilyName info 2023-08-07 13:55:45,972 INFO [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] regionserver.HStore(310): Store=3bc256268a4ab1eec2fd5f34b09fb147/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:45,974 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:45,976 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:45,982 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:45,992 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 13:55:45,993 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3bc256268a4ab1eec2fd5f34b09fb147; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59816047, jitterRate=-0.10867144167423248}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 13:55:45,993 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3bc256268a4ab1eec2fd5f34b09fb147: 2023-08-07 13:55:45,995 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147., pid=6, masterSystemTime=1691416545916 2023-08-07 13:55:45,999 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:45,999 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:46,001 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=3bc256268a4ab1eec2fd5f34b09fb147, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:46,001 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691416546000"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416546000"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416546000"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416546000"}]},"ts":"1691416546000"} 2023-08-07 13:55:46,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-07 13:55:46,007 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 3bc256268a4ab1eec2fd5f34b09fb147, server=jenkins-hbase4.apache.org,46589,1691416544393 in 243 msec 2023-08-07 13:55:46,010 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-07 13:55:46,011 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=3bc256268a4ab1eec2fd5f34b09fb147, ASSIGN in 413 msec 2023-08-07 13:55:46,012 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 13:55:46,012 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691416546012"}]},"ts":"1691416546012"} 2023-08-07 13:55:46,014 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-07 13:55:46,017 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 13:55:46,020 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 546 msec 2023-08-07 13:55:46,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/namespace 2023-08-07 13:55:46,080 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/namespace 2023-08-07 13:55:46,080 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:55:46,086 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-07 13:55:46,099 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-08-07 13:55:46,111 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 25 msec 2023-08-07 13:55:46,118 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-07 13:55:46,129 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-08-07 13:55:46,134 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-08-07 13:55:46,144 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-08-07 13:55:46,146 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-08-07 13:55:46,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.688sec 2023-08-07 13:55:46,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-07 13:55:46,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-07 13:55:46,147 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-07 13:55:46,148 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45527,1691416544159-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-07 13:55:46,148 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45527,1691416544159-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-07 13:55:46,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-07 13:55:46,156 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(139): Connect 0x149abcf1 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:46,162 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@73e83fd9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:46,164 DEBUG [hconnection-0x381b9bba-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:46,167 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59128, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:55:46,169 INFO [Listener at localhost/41247] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,45527,1691416544159 2023-08-07 13:55:46,179 DEBUG [Listener at localhost/41247] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-07 13:55:46,181 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47472, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-07 13:55:46,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 13:55:46,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-08-07 13:55:46,201 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 13:55:46,203 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 13:55:46,205 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:55:46,206 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-08-07 13:55:46,206 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3 empty. 2023-08-07 13:55:46,207 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:55:46,207 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-08-07 13:55:46,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-07 13:55:46,231 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-08-07 13:55:46,232 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => e1cfded3452bdf9174e1f916e4d8e3f3, NAME => 't1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp 2023-08-07 13:55:46,245 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:46,245 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing e1cfded3452bdf9174e1f916e4d8e3f3, disabling compactions & flushes 2023-08-07 13:55:46,245 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:55:46,246 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:55:46,246 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. after waiting 0 ms 2023-08-07 13:55:46,246 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:55:46,246 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:55:46,246 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for e1cfded3452bdf9174e1f916e4d8e3f3: 2023-08-07 13:55:46,249 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 13:55:46,251 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416546251"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691416546251"}]},"ts":"1691416546251"} 2023-08-07 13:55:46,253 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 13:55:46,255 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 13:55:46,255 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691416546255"}]},"ts":"1691416546255"} 2023-08-07 13:55:46,257 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-08-07 13:55:46,261 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 13:55:46,262 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 13:55:46,262 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 13:55:46,262 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 13:55:46,262 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=e1cfded3452bdf9174e1f916e4d8e3f3, ASSIGN}] 2023-08-07 13:55:46,264 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=e1cfded3452bdf9174e1f916e4d8e3f3, ASSIGN 2023-08-07 13:55:46,266 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=e1cfded3452bdf9174e1f916e4d8e3f3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38873,1691416540136; forceNewPlan=false, retain=false 2023-08-07 13:55:46,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-07 13:55:46,416 INFO [jenkins-hbase4:38803] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 13:55:46,418 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=e1cfded3452bdf9174e1f916e4d8e3f3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:55:46,418 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416546418"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416546418"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416546418"}]},"ts":"1691416546418"} 2023-08-07 13:55:46,420 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure e1cfded3452bdf9174e1f916e4d8e3f3, server=jenkins-hbase4.apache.org,38873,1691416540136}] 2023-08-07 13:55:46,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-07 13:55:46,546 INFO [jenkins-hbase4:46383Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:55:46,549 INFO [jenkins-hbase4:38873Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:55:46,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:55:46,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:46,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1, suffix=, logDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136, archiveDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs, maxLogs=10 2023-08-07 13:55:46,600 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK] 2023-08-07 13:55:46,600 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK] 2023-08-07 13:55:46,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1.1691416546583 2023-08-07 13:55:46,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK], DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK]] 2023-08-07 13:55:46,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e1cfded3452bdf9174e1f916e4d8e3f3, NAME => 't1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:46,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:55:46,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:46,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:55:46,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:55:46,607 INFO [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:55:46,609 DEBUG [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/cf1 2023-08-07 13:55:46,609 DEBUG [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/cf1 2023-08-07 13:55:46,610 INFO [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e1cfded3452bdf9174e1f916e4d8e3f3 columnFamilyName cf1 2023-08-07 13:55:46,611 INFO [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] regionserver.HStore(310): Store=e1cfded3452bdf9174e1f916e4d8e3f3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:46,611 INFO [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:55:46,612 DEBUG [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/norep 2023-08-07 13:55:46,613 DEBUG [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/norep 2023-08-07 13:55:46,613 INFO [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e1cfded3452bdf9174e1f916e4d8e3f3 columnFamilyName norep 2023-08-07 13:55:46,614 INFO [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] regionserver.HStore(310): Store=e1cfded3452bdf9174e1f916e4d8e3f3/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:46,616 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:55:46,617 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:55:46,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 13:55:46,621 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:55:46,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 13:55:46,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e1cfded3452bdf9174e1f916e4d8e3f3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60595100, jitterRate=-0.09706264734268188}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 13:55:46,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e1cfded3452bdf9174e1f916e4d8e3f3: 2023-08-07 13:55:46,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3., pid=11, masterSystemTime=1691416546573 2023-08-07 13:55:46,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:55:46,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:55:46,630 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=e1cfded3452bdf9174e1f916e4d8e3f3, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:55:46,630 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416546630"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416546630"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416546630"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416546630"}]},"ts":"1691416546630"} 2023-08-07 13:55:46,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-07 13:55:46,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure e1cfded3452bdf9174e1f916e4d8e3f3, server=jenkins-hbase4.apache.org,38873,1691416540136 in 214 msec 2023-08-07 13:55:46,641 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-07 13:55:46,642 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=e1cfded3452bdf9174e1f916e4d8e3f3, ASSIGN in 376 msec 2023-08-07 13:55:46,643 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 13:55:46,644 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691416546644"}]},"ts":"1691416546644"} 2023-08-07 13:55:46,648 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-08-07 13:55:46,652 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 13:55:46,654 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 457 msec 2023-08-07 13:55:46,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-07 13:55:46,828 INFO [Listener at localhost/41247] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-08-07 13:55:46,830 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 13:55:46,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-08-07 13:55:46,834 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 13:55:46,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-08-07 13:55:46,839 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 13:55:46,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-07 13:55:46,841 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:55:46,842 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe empty. 2023-08-07 13:55:46,842 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:55:46,842 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-08-07 13:55:46,863 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-08-07 13:55:46,867 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => baa9fe21942fd12dea5caf4ddaa33efe, NAME => 't2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.tmp 2023-08-07 13:55:46,892 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:46,892 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing baa9fe21942fd12dea5caf4ddaa33efe, disabling compactions & flushes 2023-08-07 13:55:46,892 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:55:46,892 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:55:46,892 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. after waiting 0 ms 2023-08-07 13:55:46,892 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:55:46,892 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:55:46,892 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for baa9fe21942fd12dea5caf4ddaa33efe: 2023-08-07 13:55:46,896 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 13:55:46,898 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416546897"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691416546897"}]},"ts":"1691416546897"} 2023-08-07 13:55:46,900 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 13:55:46,901 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 13:55:46,901 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691416546901"}]},"ts":"1691416546901"} 2023-08-07 13:55:46,903 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-08-07 13:55:46,907 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 13:55:46,909 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 13:55:46,909 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 13:55:46,909 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 13:55:46,909 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=baa9fe21942fd12dea5caf4ddaa33efe, ASSIGN}] 2023-08-07 13:55:46,912 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=baa9fe21942fd12dea5caf4ddaa33efe, ASSIGN 2023-08-07 13:55:46,913 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=baa9fe21942fd12dea5caf4ddaa33efe, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38873,1691416540136; forceNewPlan=false, retain=false 2023-08-07 13:55:46,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-07 13:55:47,064 INFO [jenkins-hbase4:38803] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 13:55:47,065 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=baa9fe21942fd12dea5caf4ddaa33efe, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:55:47,065 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416547065"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416547065"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416547065"}]},"ts":"1691416547065"} 2023-08-07 13:55:47,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure baa9fe21942fd12dea5caf4ddaa33efe, server=jenkins-hbase4.apache.org,38873,1691416540136}] 2023-08-07 13:55:47,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-07 13:55:47,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:55:47,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => baa9fe21942fd12dea5caf4ddaa33efe, NAME => 't2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:47,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:55:47,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:47,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:55:47,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:55:47,231 INFO [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:55:47,236 DEBUG [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/cf1 2023-08-07 13:55:47,236 DEBUG [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/cf1 2023-08-07 13:55:47,237 INFO [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region baa9fe21942fd12dea5caf4ddaa33efe columnFamilyName cf1 2023-08-07 13:55:47,238 INFO [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] regionserver.HStore(310): Store=baa9fe21942fd12dea5caf4ddaa33efe/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:47,238 INFO [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:55:47,241 DEBUG [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/norep 2023-08-07 13:55:47,241 DEBUG [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/norep 2023-08-07 13:55:47,242 INFO [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region baa9fe21942fd12dea5caf4ddaa33efe columnFamilyName norep 2023-08-07 13:55:47,243 INFO [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] regionserver.HStore(310): Store=baa9fe21942fd12dea5caf4ddaa33efe/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:47,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:55:47,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:55:47,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 13:55:47,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:55:47,254 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 13:55:47,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened baa9fe21942fd12dea5caf4ddaa33efe; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61024229, jitterRate=-0.09066812694072723}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 13:55:47,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for baa9fe21942fd12dea5caf4ddaa33efe: 2023-08-07 13:55:47,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe., pid=14, masterSystemTime=1691416547221 2023-08-07 13:55:47,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:55:47,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:55:47,259 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=baa9fe21942fd12dea5caf4ddaa33efe, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:55:47,259 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416547259"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416547259"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416547259"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416547259"}]},"ts":"1691416547259"} 2023-08-07 13:55:47,264 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-08-07 13:55:47,264 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure baa9fe21942fd12dea5caf4ddaa33efe, server=jenkins-hbase4.apache.org,38873,1691416540136 in 194 msec 2023-08-07 13:55:47,267 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-08-07 13:55:47,267 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=baa9fe21942fd12dea5caf4ddaa33efe, ASSIGN in 355 msec 2023-08-07 13:55:47,268 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 13:55:47,268 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691416547268"}]},"ts":"1691416547268"} 2023-08-07 13:55:47,270 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-08-07 13:55:47,273 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 13:55:47,275 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 443 msec 2023-08-07 13:55:47,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-07 13:55:47,444 INFO [Listener at localhost/41247] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-08-07 13:55:47,447 DEBUG [Listener at localhost/41247] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-07 13:55:47,449 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42098, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-07 13:55:47,452 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45527] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 13:55:47,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45527] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-08-07 13:55:47,456 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 13:55:47,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45527] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-08-07 13:55:47,457 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 13:55:47,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-07 13:55:47,460 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:47,461 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0 empty. 2023-08-07 13:55:47,461 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:47,461 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-08-07 13:55:47,489 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-08-07 13:55:47,490 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 77564e07be12a17717602214bc11ceb0, NAME => 't1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp 2023-08-07 13:55:47,504 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:47,504 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing 77564e07be12a17717602214bc11ceb0, disabling compactions & flushes 2023-08-07 13:55:47,504 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:47,504 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:47,504 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. after waiting 0 ms 2023-08-07 13:55:47,504 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:47,504 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:47,504 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 77564e07be12a17717602214bc11ceb0: 2023-08-07 13:55:47,510 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 13:55:47,512 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416547511"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691416547511"}]},"ts":"1691416547511"} 2023-08-07 13:55:47,514 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 13:55:47,515 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 13:55:47,516 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691416547515"}]},"ts":"1691416547515"} 2023-08-07 13:55:47,517 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-08-07 13:55:47,522 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 13:55:47,523 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 13:55:47,523 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 13:55:47,524 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 13:55:47,524 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-08-07 13:55:47,524 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 13:55:47,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=77564e07be12a17717602214bc11ceb0, ASSIGN}] 2023-08-07 13:55:47,526 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=77564e07be12a17717602214bc11ceb0, ASSIGN 2023-08-07 13:55:47,528 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=77564e07be12a17717602214bc11ceb0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35577,1691416544304; forceNewPlan=false, retain=false 2023-08-07 13:55:47,543 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 13:55:47,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-07 13:55:47,678 INFO [jenkins-hbase4:45527] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 13:55:47,681 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=77564e07be12a17717602214bc11ceb0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35577,1691416544304 2023-08-07 13:55:47,682 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416547681"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416547681"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416547681"}]},"ts":"1691416547681"} 2023-08-07 13:55:47,685 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 77564e07be12a17717602214bc11ceb0, server=jenkins-hbase4.apache.org,35577,1691416544304}] 2023-08-07 13:55:47,719 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-07 13:55:47,721 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-08-07 13:55:47,721 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-07 13:55:47,722 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-08-07 13:55:47,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-07 13:55:47,839 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,35577,1691416544304 2023-08-07 13:55:47,840 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 13:55:47,842 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59962, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 13:55:47,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:47,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:47,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35577%2C1691416544304.jenkins-hbase4.apache.org%2C35577%2C1691416544304.regiongroup-1, suffix=, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,35577,1691416544304, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs, maxLogs=10 2023-08-07 13:55:47,867 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:55:47,867 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK] 2023-08-07 13:55:47,867 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK] 2023-08-07 13:55:47,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,35577,1691416544304/jenkins-hbase4.apache.org%2C35577%2C1691416544304.jenkins-hbase4.apache.org%2C35577%2C1691416544304.regiongroup-1.1691416547851 2023-08-07 13:55:47,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK], DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK], DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK]] 2023-08-07 13:55:47,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 77564e07be12a17717602214bc11ceb0, NAME => 't1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:47,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:47,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:47,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:47,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:47,873 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:47,875 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1 2023-08-07 13:55:47,875 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1 2023-08-07 13:55:47,875 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 77564e07be12a17717602214bc11ceb0 columnFamilyName cf1 2023-08-07 13:55:47,876 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(310): Store=77564e07be12a17717602214bc11ceb0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:47,876 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:47,878 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/norep 2023-08-07 13:55:47,878 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/norep 2023-08-07 13:55:47,878 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 77564e07be12a17717602214bc11ceb0 columnFamilyName norep 2023-08-07 13:55:47,879 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(310): Store=77564e07be12a17717602214bc11ceb0/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:47,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:47,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:47,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 13:55:47,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:47,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 13:55:47,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 77564e07be12a17717602214bc11ceb0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60292312, jitterRate=-0.10157454013824463}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 13:55:47,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 77564e07be12a17717602214bc11ceb0: 2023-08-07 13:55:47,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0., pid=11, masterSystemTime=1691416547839 2023-08-07 13:55:47,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:47,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:47,895 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=77564e07be12a17717602214bc11ceb0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35577,1691416544304 2023-08-07 13:55:47,895 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416547895"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416547895"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416547895"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416547895"}]},"ts":"1691416547895"} 2023-08-07 13:55:47,900 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-07 13:55:47,901 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 77564e07be12a17717602214bc11ceb0, server=jenkins-hbase4.apache.org,35577,1691416544304 in 213 msec 2023-08-07 13:55:47,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-07 13:55:47,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=77564e07be12a17717602214bc11ceb0, ASSIGN in 377 msec 2023-08-07 13:55:47,904 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 13:55:47,905 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691416547904"}]},"ts":"1691416547904"} 2023-08-07 13:55:47,906 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-08-07 13:55:47,909 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 13:55:47,912 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 457 msec 2023-08-07 13:55:48,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-07 13:55:48,065 INFO [Listener at localhost/41247] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-08-07 13:55:48,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45527] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 13:55:48,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45527] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-08-07 13:55:48,072 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-08-07 13:55:48,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45527] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-08-07 13:55:48,073 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-07 13:55:48,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-07 13:55:48,075 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:48,076 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a empty. 2023-08-07 13:55:48,077 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:48,077 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-08-07 13:55:48,098 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-08-07 13:55:48,100 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 75110bd10fe307a72d13190ea044da7a, NAME => 't2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.tmp 2023-08-07 13:55:48,115 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:48,115 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 75110bd10fe307a72d13190ea044da7a, disabling compactions & flushes 2023-08-07 13:55:48,115 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:48,115 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:48,115 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. after waiting 0 ms 2023-08-07 13:55:48,116 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:48,116 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:48,116 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 75110bd10fe307a72d13190ea044da7a: 2023-08-07 13:55:48,119 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-08-07 13:55:48,120 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416548120"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1691416548120"}]},"ts":"1691416548120"} 2023-08-07 13:55:48,122 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-07 13:55:48,123 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-07 13:55:48,124 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691416548123"}]},"ts":"1691416548123"} 2023-08-07 13:55:48,125 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-08-07 13:55:48,131 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-07 13:55:48,132 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-07 13:55:48,132 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-07 13:55:48,132 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-07 13:55:48,132 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-08-07 13:55:48,132 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-07 13:55:48,132 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=75110bd10fe307a72d13190ea044da7a, ASSIGN}] 2023-08-07 13:55:48,135 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=75110bd10fe307a72d13190ea044da7a, ASSIGN 2023-08-07 13:55:48,137 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=75110bd10fe307a72d13190ea044da7a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34709,1691416544248; forceNewPlan=false, retain=false 2023-08-07 13:55:48,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-07 13:55:48,287 INFO [jenkins-hbase4:45527] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-07 13:55:48,289 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=75110bd10fe307a72d13190ea044da7a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:48,289 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416548289"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416548289"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416548289"}]},"ts":"1691416548289"} 2023-08-07 13:55:48,291 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 75110bd10fe307a72d13190ea044da7a, server=jenkins-hbase4.apache.org,34709,1691416544248}] 2023-08-07 13:55:48,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-07 13:55:48,445 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:48,446 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 13:55:48,448 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58028, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 13:55:48,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:48,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:48,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34709%2C1691416544248.jenkins-hbase4.apache.org%2C34709%2C1691416544248.regiongroup-1, suffix=, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,34709,1691416544248, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs, maxLogs=10 2023-08-07 13:55:48,469 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:55:48,469 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK] 2023-08-07 13:55:48,469 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK] 2023-08-07 13:55:48,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,34709,1691416544248/jenkins-hbase4.apache.org%2C34709%2C1691416544248.jenkins-hbase4.apache.org%2C34709%2C1691416544248.regiongroup-1.1691416548455 2023-08-07 13:55:48,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK], DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK], DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK]] 2023-08-07 13:55:48,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 75110bd10fe307a72d13190ea044da7a, NAME => 't2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:48,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:48,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:48,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:48,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:48,475 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:48,477 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1 2023-08-07 13:55:48,477 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1 2023-08-07 13:55:48,477 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75110bd10fe307a72d13190ea044da7a columnFamilyName cf1 2023-08-07 13:55:48,478 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(310): Store=75110bd10fe307a72d13190ea044da7a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:48,478 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:48,480 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/norep 2023-08-07 13:55:48,480 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/norep 2023-08-07 13:55:48,480 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75110bd10fe307a72d13190ea044da7a columnFamilyName norep 2023-08-07 13:55:48,481 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(310): Store=75110bd10fe307a72d13190ea044da7a/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:48,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:48,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:48,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 13:55:48,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:48,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-07 13:55:48,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 75110bd10fe307a72d13190ea044da7a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63051848, jitterRate=-0.06045424938201904}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 13:55:48,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 75110bd10fe307a72d13190ea044da7a: 2023-08-07 13:55:48,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a., pid=14, masterSystemTime=1691416548445 2023-08-07 13:55:48,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:48,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:48,495 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=75110bd10fe307a72d13190ea044da7a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:48,496 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416548495"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416548495"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416548495"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416548495"}]},"ts":"1691416548495"} 2023-08-07 13:55:48,501 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-08-07 13:55:48,501 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 75110bd10fe307a72d13190ea044da7a, server=jenkins-hbase4.apache.org,34709,1691416544248 in 207 msec 2023-08-07 13:55:48,504 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-08-07 13:55:48,504 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=75110bd10fe307a72d13190ea044da7a, ASSIGN in 369 msec 2023-08-07 13:55:48,505 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-07 13:55:48,505 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1691416548505"}]},"ts":"1691416548505"} 2023-08-07 13:55:48,507 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-08-07 13:55:48,511 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-08-07 13:55:48,513 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 444 msec 2023-08-07 13:55:48,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45527] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-08-07 13:55:48,678 INFO [Listener at localhost/41247] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-08-07 13:55:48,679 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(139): Connect 0x6f56592e to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:48,687 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@239a5641, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:48,688 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(139): Connect 0x303db9a9 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:48,699 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6d7f19a9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:48,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.HMaster(3464): Client=jenkins//172.31.14.131 creating replication peer, id=1, config=clusterKey=127.0.0.1:52488:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false, state=ENABLED 2023-08-07 13:55:48,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:PRE_PEER_MODIFICATION; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-08-07 13:55:48,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-08-07 13:55:48,741 INFO [PEWorker-1] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-07 13:55:48,741 INFO [PEWorker-1] zookeeper.RecoverableZooKeeper(93): Process identifier=org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@690beef0check-peer-cluster-id connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:48,746 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@690beef0check-peer-cluster-id0x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:48,747 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@690beef0check-peer-cluster-id-0x101dfb6a1a10014 connected 2023-08-07 13:55:48,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-08-07 13:55:48,849 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@690beef0check-peer-cluster-id-0x101dfb6a1a10014, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:48,858 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:55:48,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=17, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-08-07 13:55:49,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-08-07 13:55:49,038 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38873] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=16 2023-08-07 13:55:49,038 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46383] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=17 2023-08-07 13:55:49,040 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-08-07 13:55:49,040 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-08-07 13:55:49,055 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-07 13:55:49,055 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-07 13:55:49,079 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-07 13:55:49,082 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-07 13:55:49,105 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=16 2023-08-07 13:55:49,105 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=17 2023-08-07 13:55:49,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38803] master.HMaster(3641): Remote procedure done, pid=17 2023-08-07 13:55:49,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] master.HMaster(3641): Remote procedure done, pid=16 2023-08-07 13:55:49,109 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38803] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,46383,1691416540182 suceeded 2023-08-07 13:55:49,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38803] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,38873,1691416540136 suceeded 2023-08-07 13:55:49,114 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 242 msec 2023-08-07 13:55:49,115 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-08-07 13:55:49,115 INFO [PEWorker-5] replication.AddPeerProcedure(102): Successfully added ENABLED peer 1, config clusterKey=127.0.0.1:52488:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false 2023-08-07 13:55:49,115 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 242 msec 2023-08-07 13:55:49,117 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure in 399 msec 2023-08-07 13:55:49,130 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 13:55:49,130 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-08-07 13:55:49,133 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x3dea9ca3 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:49,138 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x7eb6373d to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:49,142 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@31e0b633, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:49,144 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:49,146 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@3bb9da37, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:49,148 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:49,148 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:49,150 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101dfb6a1a10017 connected 2023-08-07 13:55:49,151 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=4d10afbc-00b5-4a8d-bcc6-ae0771b33a8c to cluster=bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:55:49,153 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:49,154 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101dfb6a1a10018 connected 2023-08-07 13:55:49,155 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=4d10afbc-00b5-4a8d-bcc6-ae0771b33a8c to cluster=bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:55:49,155 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1 2023-08-07 13:55:49,155 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1 2023-08-07 13:55:49,165 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-07 13:55:49,165 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-07 13:55:49,166 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0 2023-08-07 13:55:49,166 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1 2023-08-07 13:55:49,170 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0 2023-08-07 13:55:49,170 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1 2023-08-07 13:55:49,170 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-07 13:55:49,170 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-07 13:55:49,173 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0 2023-08-07 13:55:49,174 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0 2023-08-07 13:55:49,332 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:49,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38803] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-08-07 13:55:49,336 DEBUG [Listener at localhost/41247] replication.TestReplicationSyncUpTool(91): putAndReplicateRows 2023-08-07 13:55:49,338 DEBUG [hconnection-0x5e6e4e6b-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:49,339 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-07 13:55:49,339 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-07 13:55:49,339 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-07 13:55:49,339 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:49,339 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-07 13:55:49,339 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-07 13:55:49,339 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-07 13:55:49,339 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-07 13:55:49,339 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1.1691416546583, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-07 13:55:49,339 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:49,339 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0.1691416541672, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-07 13:55:49,340 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49812, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:55:49,342 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1.1691416542713, lastWalPosition=600, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-07 13:55:49,443 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:49,454 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:49,454 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-08-07 13:55:49,471 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 6454 2023-08-07 13:55:49,472 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 58 WAL entries eligible for replication 2023-08-07 13:55:49,472 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 6454 2023-08-07 13:55:49,472 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/4=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/5=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/6=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/7=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/8=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/9=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/10=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/11=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/12=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/13=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/14=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/15=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/16=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/17=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/18=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/19=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/20=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/21=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/22=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/23=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/24=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/25=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/26=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/27=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/28=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/29=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/30=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/31=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/32=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/33=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/34=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/35=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/36=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/37=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/38=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/39=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/40=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/41=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/42=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/43=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/44=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/45=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/46=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/47=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/48=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/49=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/50=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/51=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/52=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/53=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/54=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/55=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/56=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/57=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/58=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/59=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/60=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/61=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1.1691416546583, lastWalPosition=6454, nbRowKeys=58, nbHFiles=0, heapSize=12064, lastSeqIds={}, endOfFile=false,usedBufferSize=12064] 2023-08-07 13:55:49,491 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 13:55:49,495 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52846, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-08-07 13:55:49,501 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41049] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-07 13:55:49,501 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41049] zookeeper.ReadOnlyZKClient(139): Connect 0x2120c164 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:49,508 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41049] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@36cba94a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:49,518 DEBUG [hconnection-0x70d52b0e-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:49,519 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59130, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-08-07 13:55:49,554 DEBUG [hconnection-0x70d52b0e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:49,561 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:49,567 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59966, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-08-07 13:55:49,575 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 487 2023-08-07 13:55:49,575 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-08-07 13:55:49,575 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 487 2023-08-07 13:55:49,575 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/4=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0.1691416541672, lastWalPosition=487, nbRowKeys=1, nbHFiles=0, heapSize=208, lastSeqIds={}, endOfFile=false,usedBufferSize=208] 2023-08-07 13:55:49,593 DEBUG [SinkThread-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 13:55:49,594 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:49,595 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59976, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-08-07 13:55:49,596 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35577] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-07 13:55:49,597 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35577] zookeeper.ReadOnlyZKClient(139): Connect 0x4429de86 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:49,606 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:49,606 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 42 WAL entries eligible for replication 2023-08-07 13:55:49,606 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:49,615 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35577] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@56355b37, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:49,617 DEBUG [hconnection-0x7527dc70-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:49,619 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59132, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-08-07 13:55:49,620 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41049] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-07 13:55:49,625 DEBUG [hconnection-0x7527dc70-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:49,627 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58042, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-08-07 13:55:49,630 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/62=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/63=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/64=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/65=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/66=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/67=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/68=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/69=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/70=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/71=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/72=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/73=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/74=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/75=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/76=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/77=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/78=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/79=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/80=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/81=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/82=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/83=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/84=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/85=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/86=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/87=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/88=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/89=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/90=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/91=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/92=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/93=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/94=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/95=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/96=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/97=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/98=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/99=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/100=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/101=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/102=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/103=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1.1691416546583, lastWalPosition=10975, nbRowKeys=42, nbHFiles=0, heapSize=8736, lastSeqIds={}, endOfFile=false,usedBufferSize=8736] 2023-08-07 13:55:49,632 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35577] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-07 13:55:49,635 DEBUG [SinkThread-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 13:55:49,637 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59144, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-08-07 13:55:49,639 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46589] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-07 13:55:49,640 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46589] zookeeper.ReadOnlyZKClient(139): Connect 0x5ba2d60a to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:49,657 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46589] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@5565330e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:49,667 DEBUG [hconnection-0x515abd7a-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:49,669 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59158, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-08-07 13:55:49,734 DEBUG [hconnection-0x515abd7a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:49,735 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:49,736 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:49,737 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59988, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-08-07 13:55:49,743 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:49,749 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46589] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-07 13:55:49,751 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 5614 2023-08-07 13:55:49,751 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 49 WAL entries eligible for replication 2023-08-07 13:55:49,751 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 5614 2023-08-07 13:55:49,751 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/5=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/6=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/7=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/8=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/9=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/10=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/11=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/12=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/13=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/14=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/15=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/16=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/17=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/18=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/19=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/20=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/21=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/22=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/23=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/24=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/25=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/26=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/27=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/28=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/29=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/30=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/31=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/32=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/33=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/34=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/35=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/36=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/37=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/38=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/39=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/40=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/41=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/42=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/43=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/44=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/45=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/46=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/47=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/48=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/49=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/50=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/51=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/52=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/53=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0.1691416541672, lastWalPosition=5614, nbRowKeys=49, nbHFiles=0, heapSize=10192, lastSeqIds={}, endOfFile=false,usedBufferSize=10192] 2023-08-07 13:55:49,755 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41049] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-07 13:55:49,773 DEBUG [hconnection-0x70d52b0e-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:49,776 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58048, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-08-07 13:55:49,791 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41049] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-07 13:55:49,845 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:49,850 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:49,854 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:49,884 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 11613 2023-08-07 13:55:49,885 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 57 WAL entries eligible for replication 2023-08-07 13:55:49,885 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/54=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/55=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/56=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/57=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/58=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/59=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/60=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/61=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/62=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/63=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/64=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/65=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/66=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/67=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/68=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/69=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/70=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/71=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/72=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/73=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/74=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/75=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/76=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/77=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/78=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/79=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/80=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/81=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/82=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/83=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/84=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/85=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/86=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/87=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/88=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/89=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/90=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/91=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/92=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/93=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/94=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/95=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/96=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/97=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/98=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/99=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/100=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/101=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/102=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/103=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/104=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/105=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/106=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/107=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/108=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/109=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/110=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0.1691416541672, lastWalPosition=11613, nbRowKeys=57, nbHFiles=0, heapSize=11856, lastSeqIds={}, endOfFile=false,usedBufferSize=11856] 2023-08-07 13:55:49,886 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 11613 2023-08-07 13:55:49,898 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35577] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-07 13:55:49,911 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35577] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-07 13:55:49,955 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:49,965 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:49,967 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:49,968 INFO [jenkins-hbase4:34709Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:55:49,972 INFO [jenkins-hbase4:46589Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:55:49,972 INFO [jenkins-hbase4:46589Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 42 2023-08-07 13:55:49,982 INFO [jenkins-hbase4:41049Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:55:49,982 INFO [jenkins-hbase4:41049Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 107 2023-08-07 13:55:49,985 INFO [jenkins-hbase4:35577Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:55:49,985 INFO [jenkins-hbase4:35577Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 58 2023-08-07 13:55:50,006 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 18832 2023-08-07 13:55:50,007 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 67 WAL entries eligible for replication 2023-08-07 13:55:50,007 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/111=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/112=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/113=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/114=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/115=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/116=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/117=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/118=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/119=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/120=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/121=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/122=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/123=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/124=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/125=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/126=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/127=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/128=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/129=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/130=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/131=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/132=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/133=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/134=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/135=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/136=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/137=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/138=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/139=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/140=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/141=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/142=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/143=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/144=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/145=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/146=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/147=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/148=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/149=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/150=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/151=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/152=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/153=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/154=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/155=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/156=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/157=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/158=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/159=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/160=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/161=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/162=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/163=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/164=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/165=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/166=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/167=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/168=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/169=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/170=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/171=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/172=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/173=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/174=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/175=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/176=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/177=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0.1691416541672, lastWalPosition=18832, nbRowKeys=67, nbHFiles=0, heapSize=13936, lastSeqIds={}, endOfFile=false,usedBufferSize=13936] 2023-08-07 13:55:50,008 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 18832 2023-08-07 13:55:50,012 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46589] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-07 13:55:50,018 DEBUG [hconnection-0x515abd7a-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:50,020 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58058, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-08-07 13:55:50,033 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46589] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-07 13:55:50,064 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:50,073 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:50,077 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:50,120 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:50,120 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 26 WAL entries eligible for replication 2023-08-07 13:55:50,121 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:50,121 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/178=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/179=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/180=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/181=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/182=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/183=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/184=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/185=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/186=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/187=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/188=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/189=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/190=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/191=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/192=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/193=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/194=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/195=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/196=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/197=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/198=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/199=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/200=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/201=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/202=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/203=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0.1691416541672, lastWalPosition=21752, nbRowKeys=26, nbHFiles=0, heapSize=5408, lastSeqIds={}, endOfFile=false,usedBufferSize=5408] 2023-08-07 13:55:50,123 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35577] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-07 13:55:50,128 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35577] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-07 13:55:50,176 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:50,185 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:50,186 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:50,227 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:50,283 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:50,294 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:50,294 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:50,334 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:50,390 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:50,402 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:50,403 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:50,447 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:50,496 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:50,507 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:50,508 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:50,547 DEBUG [hconnection-0x1c266219-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:50,549 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37484, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:55:50,555 DEBUG [hconnection-0x1c266219-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:50,556 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:50,557 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59806, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:55:50,573 DEBUG [hconnection-0x1c266219-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:50,575 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52718, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:55:50,582 DEBUG [Listener at localhost/41247] replication.TestReplicationSyncUpTool(145): mimicSyncUpAfterDelete 2023-08-07 13:55:50,582 INFO [Listener at localhost/41247] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-07 13:55:50,583 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x149abcf1 to 127.0.0.1:52488 2023-08-07 13:55:50,583 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:50,584 DEBUG [Listener at localhost/41247] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-07 13:55:50,584 DEBUG [Listener at localhost/41247] util.JVMClusterUtil(257): Found active master hash=1952825894, stopped=false 2023-08-07 13:55:50,584 INFO [Listener at localhost/41247] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,45527,1691416544159 2023-08-07 13:55:50,585 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-07 13:55:50,585 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:41049-0x101dfb6a1a1000a, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-07 13:55:50,586 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-07 13:55:50,585 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-07 13:55:50,585 INFO [Listener at localhost/41247] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 13:55:50,585 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-07 13:55:50,586 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:55:50,586 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x17cec3ba to 127.0.0.1:52488 2023-08-07 13:55:50,589 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:50,590 INFO [Listener at localhost/41247] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34709,1691416544248' ***** 2023-08-07 13:55:50,590 INFO [Listener at localhost/41247] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 13:55:50,590 INFO [Listener at localhost/41247] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,35577,1691416544304' ***** 2023-08-07 13:55:50,590 INFO [Listener at localhost/41247] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 13:55:50,590 INFO [RS:0;jenkins-hbase4:34709] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 13:55:50,590 INFO [Listener at localhost/41247] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41049,1691416544342' ***** 2023-08-07 13:55:50,590 INFO [Listener at localhost/41247] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 13:55:50,590 INFO [Listener at localhost/41247] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46589,1691416544393' ***** 2023-08-07 13:55:50,591 INFO [Listener at localhost/41247] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 13:55:50,591 INFO [RS:2;jenkins-hbase4:41049] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 13:55:50,591 INFO [RS:3;jenkins-hbase4:46589] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 13:55:50,591 INFO [RS:2;jenkins-hbase4:41049] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 13:55:50,591 INFO [RS:3;jenkins-hbase4:46589] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 13:55:50,591 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 13:55:50,590 INFO [RS:1;jenkins-hbase4:35577] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 13:55:50,592 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 13:55:50,592 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 13:55:50,593 INFO [RS:1;jenkins-hbase4:35577] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 13:55:50,591 INFO [RS:3;jenkins-hbase4:46589] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 13:55:50,591 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 13:55:50,591 INFO [RS:2;jenkins-hbase4:41049] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 13:55:50,590 INFO [RS:0;jenkins-hbase4:34709] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 13:55:50,595 INFO [RS:2;jenkins-hbase4:41049] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41049,1691416544342 2023-08-07 13:55:50,595 INFO [RS:0;jenkins-hbase4:34709] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 13:55:50,595 INFO [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(3305): Received CLOSE for 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:50,594 INFO [RS:1;jenkins-hbase4:35577] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 13:55:50,595 INFO [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(3305): Received CLOSE for 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:50,595 DEBUG [RS:2;jenkins-hbase4:41049] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x013efa24 to 127.0.0.1:52488 2023-08-07 13:55:50,595 INFO [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(3305): Received CLOSE for 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:50,595 DEBUG [RS:2;jenkins-hbase4:41049] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:50,596 INFO [RS:2;jenkins-hbase4:41049] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41049,1691416544342; all regions closed. 2023-08-07 13:55:50,596 INFO [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:50,596 DEBUG [RS:3;jenkins-hbase4:46589] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4e7ae206 to 127.0.0.1:52488 2023-08-07 13:55:50,596 DEBUG [RS:3;jenkins-hbase4:46589] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:50,596 INFO [RS:3;jenkins-hbase4:46589] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 13:55:50,596 INFO [RS:3;jenkins-hbase4:46589] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 13:55:50,596 INFO [RS:3;jenkins-hbase4:46589] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 13:55:50,596 INFO [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-07 13:55:50,596 INFO [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,35577,1691416544304 2023-08-07 13:55:50,596 DEBUG [RS:1;jenkins-hbase4:35577] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3970f1d4 to 127.0.0.1:52488 2023-08-07 13:55:50,597 DEBUG [RS:1;jenkins-hbase4:35577] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:50,597 INFO [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-08-07 13:55:50,597 DEBUG [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(1478): Online Regions={77564e07be12a17717602214bc11ceb0=t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.} 2023-08-07 13:55:50,598 DEBUG [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(1504): Waiting on 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:50,598 INFO [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:50,599 DEBUG [RS:0;jenkins-hbase4:34709] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x473cb62f to 127.0.0.1:52488 2023-08-07 13:55:50,599 DEBUG [RS:0;jenkins-hbase4:34709] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:50,599 INFO [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-08-07 13:55:50,599 DEBUG [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(1478): Online Regions={75110bd10fe307a72d13190ea044da7a=t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.} 2023-08-07 13:55:50,599 DEBUG [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(1504): Waiting on 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:50,599 INFO [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-08-07 13:55:50,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 75110bd10fe307a72d13190ea044da7a, disabling compactions & flushes 2023-08-07 13:55:50,600 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:55:50,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:50,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 77564e07be12a17717602214bc11ceb0, disabling compactions & flushes 2023-08-07 13:55:50,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:50,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:50,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. after waiting 0 ms 2023-08-07 13:55:50,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:50,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3bc256268a4ab1eec2fd5f34b09fb147, disabling compactions & flushes 2023-08-07 13:55:50,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:50,602 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 13:55:50,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. after waiting 0 ms 2023-08-07 13:55:50,601 DEBUG [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, 3bc256268a4ab1eec2fd5f34b09fb147=hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.} 2023-08-07 13:55:50,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:50,603 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 13:55:50,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:50,603 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 13:55:50,603 DEBUG [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(1504): Waiting on 1588230740, 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:50,603 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 75110bd10fe307a72d13190ea044da7a 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-08-07 13:55:50,603 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 77564e07be12a17717602214bc11ceb0 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-08-07 13:55:50,603 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 13:55:50,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:50,603 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 13:55:50,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. after waiting 0 ms 2023-08-07 13:55:50,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:50,603 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 3bc256268a4ab1eec2fd5f34b09fb147 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-07 13:55:50,603 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.65 KB heapSize=7.06 KB 2023-08-07 13:55:50,619 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:50,619 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:50,619 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:50,625 DEBUG [RS:2;jenkins-hbase4:41049] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs 2023-08-07 13:55:50,625 INFO [RS:2;jenkins-hbase4:41049] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41049%2C1691416544342.jenkins-hbase4.apache.org%2C41049%2C1691416544342.regiongroup-0:(num 1691416545093) 2023-08-07 13:55:50,625 DEBUG [RS:2;jenkins-hbase4:41049] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:50,625 INFO [RS:2;jenkins-hbase4:41049] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:55:50,625 INFO [RS:2;jenkins-hbase4:41049] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-07 13:55:50,626 INFO [RS:2;jenkins-hbase4:41049] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 13:55:50,626 INFO [RS:2;jenkins-hbase4:41049] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 13:55:50,626 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 13:55:50,626 INFO [RS:2;jenkins-hbase4:41049] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 13:55:50,626 DEBUG [RS:2;jenkins-hbase4:41049] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2120c164 to 127.0.0.1:52488 2023-08-07 13:55:50,626 DEBUG [RS:2;jenkins-hbase4:41049] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:50,628 INFO [RS:2;jenkins-hbase4:41049] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41049 2023-08-07 13:55:50,663 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:50,674 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:55:50,674 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:55:50,676 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:55:50,691 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:41049-0x101dfb6a1a1000a, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,41049,1691416544342 2023-08-07 13:55:50,691 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,41049,1691416544342 2023-08-07 13:55:50,691 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,41049,1691416544342 2023-08-07 13:55:50,691 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:50,691 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:50,692 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,41049,1691416544342 2023-08-07 13:55:50,692 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:50,691 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:50,691 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:41049-0x101dfb6a1a1000a, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:50,691 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a10018, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:50,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41049-0x101dfb6a1a1000a, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:55:50,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:55:50,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:55:50,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:55:50,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:55:50,694 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41049,1691416544342] 2023-08-07 13:55:50,694 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41049,1691416544342; numProcessing=1 2023-08-07 13:55:50,695 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,41049,1691416544342 already deleted, retry=false 2023-08-07 13:55:50,696 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41049,1691416544342 expired; onlineServers=3 2023-08-07 13:55:50,699 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-07 13:55:50,721 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=9 (bloomFilter=true), to=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/.tmp/cf1/8a858ad05b094550b24538261ebbea4e 2023-08-07 13:55:50,726 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/.tmp/info/8766d5bf2c17410e8580146469a8662e 2023-08-07 13:55:50,726 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:50,726 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:50,728 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/.tmp/info/2db3ce6e71eb4d9a95061ddf2a1afa69 2023-08-07 13:55:50,728 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:50,767 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:50,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/.tmp/cf1/8a858ad05b094550b24538261ebbea4e as hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1/8a858ad05b094550b24538261ebbea4e 2023-08-07 13:55:50,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/.tmp/info/2db3ce6e71eb4d9a95061ddf2a1afa69 as hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/info/2db3ce6e71eb4d9a95061ddf2a1afa69 2023-08-07 13:55:50,782 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1/8a858ad05b094550b24538261ebbea4e, entries=200, sequenceid=9, filesize=12.2 K 2023-08-07 13:55:50,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/info/2db3ce6e71eb4d9a95061ddf2a1afa69, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 13:55:50,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 3bc256268a4ab1eec2fd5f34b09fb147 in 188ms, sequenceid=6, compaction requested=false 2023-08-07 13:55:50,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 75110bd10fe307a72d13190ea044da7a in 188ms, sequenceid=9, compaction requested=false 2023-08-07 13:55:50,794 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/.tmp/table/b6b6cd7163ac41e2b3c4c475d90c98bf 2023-08-07 13:55:50,798 DEBUG [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(1504): Waiting on 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:50,799 DEBUG [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(1504): Waiting on 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:50,802 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-08-07 13:55:50,802 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-07 13:55:50,803 DEBUG [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(1504): Waiting on 1588230740, 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:50,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:50,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:50,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3bc256268a4ab1eec2fd5f34b09fb147: 2023-08-07 13:55:50,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 75110bd10fe307a72d13190ea044da7a: 2023-08-07 13:55:50,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:50,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:50,807 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/.tmp/info/8766d5bf2c17410e8580146469a8662e as hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info/8766d5bf2c17410e8580146469a8662e 2023-08-07 13:55:50,813 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info/8766d5bf2c17410e8580146469a8662e, entries=30, sequenceid=19, filesize=8.1 K 2023-08-07 13:55:50,814 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/.tmp/table/b6b6cd7163ac41e2b3c4c475d90c98bf as hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/table/b6b6cd7163ac41e2b3c4c475d90c98bf 2023-08-07 13:55:50,820 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/table/b6b6cd7163ac41e2b3c4c475d90c98bf, entries=6, sequenceid=19, filesize=4.8 K 2023-08-07 13:55:50,821 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.65 KB/3738, heapSize ~6.78 KB/6944, currentSize=0 B/0 for 1588230740 in 218ms, sequenceid=19, compaction requested=false 2023-08-07 13:55:50,833 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-08-07 13:55:50,834 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 13:55:50,834 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:50,834 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:50,834 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 13:55:50,835 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 13:55:50,835 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-07 13:55:50,837 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:50,873 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:50,876 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-07 13:55:50,877 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-07 13:55:50,899 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:41049-0x101dfb6a1a1000a, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:50,899 INFO [RS:2;jenkins-hbase4:41049] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41049,1691416544342; zookeeper connection closed. 2023-08-07 13:55:50,899 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:41049-0x101dfb6a1a1000a, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:50,901 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@19d9036] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@19d9036 2023-08-07 13:55:50,916 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-07 13:55:50,916 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-07 13:55:50,923 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-07 13:55:50,923 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-07 13:55:50,939 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:50,939 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:50,941 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:50,978 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:50,999 DEBUG [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(1504): Waiting on 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:51,001 INFO [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34709,1691416544248; all regions closed. 2023-08-07 13:55:51,004 INFO [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46589,1691416544393; all regions closed. 2023-08-07 13:55:51,010 DEBUG [RS:0;jenkins-hbase4:34709] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs 2023-08-07 13:55:51,010 INFO [RS:0;jenkins-hbase4:34709] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34709%2C1691416544248.jenkins-hbase4.apache.org%2C34709%2C1691416544248.regiongroup-1:(num 1691416548455) 2023-08-07 13:55:51,011 DEBUG [RS:3;jenkins-hbase4:46589] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs 2023-08-07 13:55:51,011 INFO [RS:3;jenkins-hbase4:46589] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46589%2C1691416544393.meta:.meta(num 1691416545340) 2023-08-07 13:55:51,022 DEBUG [RS:0;jenkins-hbase4:34709] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs 2023-08-07 13:55:51,022 INFO [RS:0;jenkins-hbase4:34709] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34709%2C1691416544248.jenkins-hbase4.apache.org%2C34709%2C1691416544248.regiongroup-0:(num 1691416545085) 2023-08-07 13:55:51,022 DEBUG [RS:3;jenkins-hbase4:46589] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs 2023-08-07 13:55:51,022 DEBUG [RS:0;jenkins-hbase4:34709] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:51,022 INFO [RS:3;jenkins-hbase4:46589] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46589%2C1691416544393.jenkins-hbase4.apache.org%2C46589%2C1691416544393.regiongroup-1:(num 1691416545928) 2023-08-07 13:55:51,022 INFO [RS:0;jenkins-hbase4:34709] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:55:51,022 INFO [RS:0;jenkins-hbase4:34709] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-07 13:55:51,023 INFO [RS:0;jenkins-hbase4:34709] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 13:55:51,023 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 13:55:51,023 INFO [RS:0;jenkins-hbase4:34709] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 13:55:51,024 INFO [RS:0;jenkins-hbase4:34709] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 13:55:51,025 INFO [RS:0;jenkins-hbase4:34709] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34709 2023-08-07 13:55:51,032 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:51,033 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:51,033 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:51,033 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,34709,1691416544248 2023-08-07 13:55:51,033 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a10018, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:51,036 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-07 13:55:51,037 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34709,1691416544248] 2023-08-07 13:55:51,037 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34709,1691416544248; numProcessing=2 2023-08-07 13:55:51,039 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,34709,1691416544248 already deleted, retry=false 2023-08-07 13:55:51,039 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34709,1691416544248 expired; onlineServers=2 2023-08-07 13:55:51,042 DEBUG [RS:3;jenkins-hbase4:46589] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs 2023-08-07 13:55:51,042 INFO [RS:3;jenkins-hbase4:46589] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46589%2C1691416544393.jenkins-hbase4.apache.org%2C46589%2C1691416544393.regiongroup-0:(num 1691416545083) 2023-08-07 13:55:51,042 DEBUG [RS:3;jenkins-hbase4:46589] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:51,042 INFO [RS:3;jenkins-hbase4:46589] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:55:51,042 INFO [RS:3;jenkins-hbase4:46589] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-07 13:55:51,042 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 13:55:51,043 DEBUG [RS:3;jenkins-hbase4:46589] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5ba2d60a to 127.0.0.1:52488 2023-08-07 13:55:51,043 DEBUG [RS:3;jenkins-hbase4:46589] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:51,045 INFO [RS:3;jenkins-hbase4:46589] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46589 2023-08-07 13:55:51,047 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:51,048 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:51,048 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:51,048 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:51,048 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a10018, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:51,048 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,46589,1691416544393 2023-08-07 13:55:51,048 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:51,048 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-07 13:55:51,050 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46589,1691416544393] 2023-08-07 13:55:51,050 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46589,1691416544393; numProcessing=3 2023-08-07 13:55:51,051 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46589,1691416544393 already deleted, retry=false 2023-08-07 13:55:51,051 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46589,1691416544393 expired; onlineServers=1 2023-08-07 13:55:51,084 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:51,124 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/.tmp/cf1/ed63a5fa2177464491100ded6dbb022b 2023-08-07 13:55:51,131 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/.tmp/cf1/ed63a5fa2177464491100ded6dbb022b as hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1/ed63a5fa2177464491100ded6dbb022b 2023-08-07 13:55:51,138 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1/ed63a5fa2177464491100ded6dbb022b, entries=100, sequenceid=6, filesize=8.3 K 2023-08-07 13:55:51,140 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 77564e07be12a17717602214bc11ceb0 in 537ms, sequenceid=6, compaction requested=false 2023-08-07 13:55:51,147 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-07 13:55:51,148 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:51,148 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 77564e07be12a17717602214bc11ceb0: 2023-08-07 13:55:51,149 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:51,153 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:51,153 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:51,154 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:51,189 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:51,199 INFO [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,35577,1691416544304; all regions closed. 2023-08-07 13:55:51,207 DEBUG [RS:1;jenkins-hbase4:35577] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs 2023-08-07 13:55:51,207 INFO [RS:1;jenkins-hbase4:35577] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35577%2C1691416544304.jenkins-hbase4.apache.org%2C35577%2C1691416544304.regiongroup-0:(num 1691416545094) 2023-08-07 13:55:51,212 DEBUG [RS:1;jenkins-hbase4:35577] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs 2023-08-07 13:55:51,212 INFO [RS:1;jenkins-hbase4:35577] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35577%2C1691416544304.jenkins-hbase4.apache.org%2C35577%2C1691416544304.regiongroup-1:(num 1691416547851) 2023-08-07 13:55:51,212 DEBUG [RS:1;jenkins-hbase4:35577] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:51,213 INFO [RS:1;jenkins-hbase4:35577] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:55:51,213 INFO [RS:1;jenkins-hbase4:35577] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-07 13:55:51,213 INFO [RS:1;jenkins-hbase4:35577] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 13:55:51,213 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 13:55:51,213 INFO [RS:1;jenkins-hbase4:35577] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 13:55:51,213 INFO [RS:1;jenkins-hbase4:35577] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 13:55:51,213 DEBUG [RS:1;jenkins-hbase4:35577] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4429de86 to 127.0.0.1:52488 2023-08-07 13:55:51,213 DEBUG [RS:1;jenkins-hbase4:35577] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:51,214 INFO [RS:1;jenkins-hbase4:35577] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35577 2023-08-07 13:55:51,219 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:51,219 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a10018, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:51,219 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,35577,1691416544304 2023-08-07 13:55:51,219 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-07 13:55:51,220 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,35577,1691416544304] 2023-08-07 13:55:51,221 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,35577,1691416544304; numProcessing=4 2023-08-07 13:55:51,222 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,35577,1691416544304 already deleted, retry=false 2023-08-07 13:55:51,222 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,35577,1691416544304 expired; onlineServers=0 2023-08-07 13:55:51,222 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45527,1691416544159' ***** 2023-08-07 13:55:51,222 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-07 13:55:51,223 DEBUG [M:0;jenkins-hbase4:45527] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@705a4872, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 13:55:51,223 INFO [M:0;jenkins-hbase4:45527] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45527,1691416544159 2023-08-07 13:55:51,223 INFO [M:0;jenkins-hbase4:45527] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45527,1691416544159; all regions closed. 2023-08-07 13:55:51,223 DEBUG [M:0;jenkins-hbase4:45527] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:51,223 DEBUG [M:0;jenkins-hbase4:45527] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-07 13:55:51,223 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-07 13:55:51,223 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416544735] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416544735,5,FailOnTimeoutGroup] 2023-08-07 13:55:51,223 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416544735] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416544735,5,FailOnTimeoutGroup] 2023-08-07 13:55:51,223 DEBUG [M:0;jenkins-hbase4:45527] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-07 13:55:51,225 INFO [M:0;jenkins-hbase4:45527] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-07 13:55:51,225 INFO [M:0;jenkins-hbase4:45527] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-07 13:55:51,225 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-08-07 13:55:51,225 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:55:51,225 INFO [M:0;jenkins-hbase4:45527] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-07 13:55:51,225 DEBUG [M:0;jenkins-hbase4:45527] master.HMaster(1512): Stopping service threads 2023-08-07 13:55:51,225 INFO [M:0;jenkins-hbase4:45527] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-07 13:55:51,226 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-07 13:55:51,226 ERROR [M:0;jenkins-hbase4:45527] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-08-07 13:55:51,227 INFO [M:0;jenkins-hbase4:45527] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 13:55:51,227 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-07 13:55:51,227 DEBUG [M:0;jenkins-hbase4:45527] zookeeper.ZKUtil(398): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-08-07 13:55:51,227 WARN [M:0;jenkins-hbase4:45527] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-07 13:55:51,227 INFO [M:0;jenkins-hbase4:45527] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 13:55:51,228 INFO [M:0;jenkins-hbase4:45527] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-07 13:55:51,229 DEBUG [M:0;jenkins-hbase4:45527] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 13:55:51,229 INFO [M:0;jenkins-hbase4:45527] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:51,229 DEBUG [M:0;jenkins-hbase4:45527] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:51,229 DEBUG [M:0;jenkins-hbase4:45527] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 13:55:51,229 DEBUG [M:0;jenkins-hbase4:45527] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:51,229 INFO [M:0;jenkins-hbase4:45527] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=55.39 KB heapSize=65.18 KB 2023-08-07 13:55:51,245 INFO [M:0;jenkins-hbase4:45527] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=55.39 KB at sequenceid=116 (bloomFilter=true), to=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5eec9c061c4641719002e9ed31c10ef2 2023-08-07 13:55:51,252 DEBUG [M:0;jenkins-hbase4:45527] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5eec9c061c4641719002e9ed31c10ef2 as hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5eec9c061c4641719002e9ed31c10ef2 2023-08-07 13:55:51,258 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:51,258 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:51,260 INFO [M:0;jenkins-hbase4:45527] regionserver.HStore(1080): Added hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5eec9c061c4641719002e9ed31c10ef2, entries=14, sequenceid=116, filesize=8.1 K 2023-08-07 13:55:51,260 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:51,261 INFO [M:0;jenkins-hbase4:45527] regionserver.HRegion(2948): Finished flush of dataSize ~55.39 KB/56720, heapSize ~65.16 KB/66728, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 32ms, sequenceid=116, compaction requested=false 2023-08-07 13:55:51,264 INFO [M:0;jenkins-hbase4:45527] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:51,264 DEBUG [M:0;jenkins-hbase4:45527] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 13:55:51,270 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 13:55:51,270 INFO [M:0;jenkins-hbase4:45527] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-07 13:55:51,271 INFO [M:0;jenkins-hbase4:45527] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45527 2023-08-07 13:55:51,273 DEBUG [M:0;jenkins-hbase4:45527] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,45527,1691416544159 already deleted, retry=false 2023-08-07 13:55:51,293 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:51,364 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:51,364 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:51,365 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:51,398 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:51,469 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:51,469 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:51,469 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:51,500 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:51,500 INFO [M:0;jenkins-hbase4:45527] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45527,1691416544159; zookeeper connection closed. 2023-08-07 13:55:51,501 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:45527-0x101dfb6a1a10007, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:51,505 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:51,546 INFO [jenkins-hbase4:46383Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1]: currently replicating from: hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1.1691416542713 at position: 600 walGroup [jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0]: currently replicating from: hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0.1691416541682 at position: -1 2023-08-07 13:55:51,549 INFO [jenkins-hbase4:38873Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 300, current progress: walGroup [jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1]: currently replicating from: hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1.1691416546583 at position: 10975 walGroup [jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0]: currently replicating from: hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0.1691416541672 at position: 21752 2023-08-07 13:55:51,575 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:51,575 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:51,575 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:51,601 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:51,601 INFO [RS:1;jenkins-hbase4:35577] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,35577,1691416544304; zookeeper connection closed. 2023-08-07 13:55:51,601 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:35577-0x101dfb6a1a10009, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:51,602 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5e007244] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5e007244 2023-08-07 13:55:51,610 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:51,681 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:51,681 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:51,681 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:51,715 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:51,786 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:51,787 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:51,787 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:51,801 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:51,801 INFO [RS:3;jenkins-hbase4:46589] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46589,1691416544393; zookeeper connection closed. 2023-08-07 13:55:51,801 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46589-0x101dfb6a1a1000b, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:51,803 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@72bdff1a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@72bdff1a 2023-08-07 13:55:51,820 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:51,893 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:51,893 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:51,893 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:51,926 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-08-07 13:55:51,999 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-08-07 13:55:52,000 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-08-07 13:55:52,001 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:52,001 INFO [RS:0;jenkins-hbase4:34709] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34709,1691416544248; zookeeper connection closed. 2023-08-07 13:55:52,001 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34709-0x101dfb6a1a10008, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:52,003 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4cc457dc] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4cc457dc 2023-08-07 13:55:52,003 INFO [Listener at localhost/41247] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 4 regionserver(s) complete 2023-08-07 13:55:52,004 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:52,031 INFO [Listener at localhost/41247] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-07 13:55:52,031 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1aa5818b to 127.0.0.1:52488 2023-08-07 13:55:52,032 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:52,032 DEBUG [Listener at localhost/41247] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-07 13:55:52,032 DEBUG [Listener at localhost/41247] util.JVMClusterUtil(257): Found active master hash=1549481926, stopped=false 2023-08-07 13:55:52,032 INFO [Listener at localhost/41247] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38803,1691416538961 2023-08-07 13:55:52,034 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-07 13:55:52,034 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-07 13:55:52,034 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:46383-0x101dfb6a1a10002, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-07 13:55:52,034 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:55:52,034 INFO [Listener at localhost/41247] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 13:55:52,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-07 13:55:52,035 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0a122ee6 to 127.0.0.1:52488 2023-08-07 13:55:52,035 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:52,035 INFO [Listener at localhost/41247] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38873,1691416540136' ***** 2023-08-07 13:55:52,035 INFO [Listener at localhost/41247] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 13:55:52,035 INFO [Listener at localhost/41247] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46383,1691416540182' ***** 2023-08-07 13:55:52,035 INFO [Listener at localhost/41247] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 13:55:52,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-07 13:55:52,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46383-0x101dfb6a1a10002, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-07 13:55:52,042 INFO [RS:0;jenkins-hbase4:38873] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 13:55:52,042 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 13:55:52,042 INFO [RS:1;jenkins-hbase4:46383] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 13:55:52,042 INFO [RS:0;jenkins-hbase4:38873] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 13:55:52,042 INFO [RS:1;jenkins-hbase4:46383] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 13:55:52,042 INFO [RS:0;jenkins-hbase4:38873] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 13:55:52,043 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 13:55:52,044 INFO [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(3305): Received CLOSE for e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:55:52,043 INFO [RS:1;jenkins-hbase4:46383] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 13:55:52,045 INFO [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(3305): Received CLOSE for baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:55:52,045 INFO [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(3305): Received CLOSE for ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:55:52,045 INFO [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:55:52,045 DEBUG [RS:0;jenkins-hbase4:38873] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x29ed6361 to 127.0.0.1:52488 2023-08-07 13:55:52,045 DEBUG [RS:0;jenkins-hbase4:38873] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:52,045 INFO [RS:0;jenkins-hbase4:38873] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 13:55:52,045 INFO [RS:0;jenkins-hbase4:38873] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 13:55:52,045 INFO [RS:0;jenkins-hbase4:38873] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 13:55:52,045 INFO [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-07 13:55:52,045 INFO [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:55:52,046 INFO [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-08-07 13:55:52,046 DEBUG [RS:1;jenkins-hbase4:46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1d3e587a to 127.0.0.1:52488 2023-08-07 13:55:52,046 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e1cfded3452bdf9174e1f916e4d8e3f3, disabling compactions & flushes 2023-08-07 13:55:52,047 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:55:52,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:55:52,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. after waiting 0 ms 2023-08-07 13:55:52,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:55:52,048 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing e1cfded3452bdf9174e1f916e4d8e3f3 2/2 column families, dataSize=6.25 KB heapSize=21.71 KB 2023-08-07 13:55:52,048 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28400 2023-08-07 13:55:52,048 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-08-07 13:55:52,048 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28400 2023-08-07 13:55:52,049 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0.1691416541672, lastWalPosition=28400, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=false,usedBufferSize=8120] 2023-08-07 13:55:52,047 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 13:55:52,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ee0eecdf41bc57a826e75999fe370d6d, disabling compactions & flushes 2023-08-07 13:55:52,046 DEBUG [RS:1;jenkins-hbase4:46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:52,046 DEBUG [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, e1cfded3452bdf9174e1f916e4d8e3f3=t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3., baa9fe21942fd12dea5caf4ddaa33efe=t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe.} 2023-08-07 13:55:52,050 INFO [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-08-07 13:55:52,050 DEBUG [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(1504): Waiting on 1588230740, baa9fe21942fd12dea5caf4ddaa33efe, e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:55:52,050 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:55:52,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:55:52,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. after waiting 0 ms 2023-08-07 13:55:52,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:55:52,050 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ee0eecdf41bc57a826e75999fe370d6d 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-07 13:55:52,050 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 13:55:52,050 DEBUG [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(1478): Online Regions={ee0eecdf41bc57a826e75999fe370d6d=hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d.} 2023-08-07 13:55:52,050 DEBUG [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(1504): Waiting on ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:55:52,051 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 13:55:52,051 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 13:55:52,051 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 13:55:52,051 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.87 KB heapSize=7.44 KB 2023-08-07 13:55:52,074 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:55:52,077 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:55:52,081 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/.tmp/info/2a6ffe12094b4b58a67fa9037378333e 2023-08-07 13:55:52,091 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.76 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/.tmp/cf1/7d95de56b4fb4170b2c83aba3569728b 2023-08-07 13:55:52,105 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7d95de56b4fb4170b2c83aba3569728b 2023-08-07 13:55:52,111 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 845 2023-08-07 13:55:52,111 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-07 13:55:52,112 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 845 2023-08-07 13:55:52,112 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1.1691416542713, lastWalPosition=845, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-07 13:55:52,112 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:52,112 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-08-07 13:55:52,112 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-08-07 13:55:52,113 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-08-07 13:55:52,113 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1.1691416546583, lastWalPosition=14562, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=false,usedBufferSize=4120] 2023-08-07 13:55:52,116 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d/.tmp/info/0585593dc6864458ab6dab9850300c9f 2023-08-07 13:55:52,130 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d/.tmp/info/0585593dc6864458ab6dab9850300c9f as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d/info/0585593dc6864458ab6dab9850300c9f 2023-08-07 13:55:52,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d/info/0585593dc6864458ab6dab9850300c9f, entries=2, sequenceid=6, filesize=4.8 K 2023-08-07 13:55:52,140 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for ee0eecdf41bc57a826e75999fe370d6d in 90ms, sequenceid=6, compaction requested=false 2023-08-07 13:55:52,158 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/.tmp/rep_barrier/73c4a69ba56644f092aded864ae1f0b9 2023-08-07 13:55:52,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-07 13:55:52,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:55:52,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.50 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/.tmp/norep/af3a3d1432f64ef9a61da51970b3a8cd 2023-08-07 13:55:52,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ee0eecdf41bc57a826e75999fe370d6d: 2023-08-07 13:55:52,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:55:52,167 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28400 2023-08-07 13:55:52,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for af3a3d1432f64ef9a61da51970b3a8cd 2023-08-07 13:55:52,176 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/.tmp/cf1/7d95de56b4fb4170b2c83aba3569728b as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/cf1/7d95de56b4fb4170b2c83aba3569728b 2023-08-07 13:55:52,187 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7d95de56b4fb4170b2c83aba3569728b 2023-08-07 13:55:52,187 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/cf1/7d95de56b4fb4170b2c83aba3569728b, entries=100, sequenceid=106, filesize=8.2 K 2023-08-07 13:55:52,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/.tmp/norep/af3a3d1432f64ef9a61da51970b3a8cd as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/norep/af3a3d1432f64ef9a61da51970b3a8cd 2023-08-07 13:55:52,191 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/.tmp/table/1381c4f6b52d44928376e105bf7815d4 2023-08-07 13:55:52,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for af3a3d1432f64ef9a61da51970b3a8cd 2023-08-07 13:55:52,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/norep/af3a3d1432f64ef9a61da51970b3a8cd, entries=51, sequenceid=106, filesize=6.5 K 2023-08-07 13:55:52,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~6.25 KB/6401, heapSize ~21.68 KB/22200, currentSize=0 B/0 for e1cfded3452bdf9174e1f916e4d8e3f3 in 151ms, sequenceid=106, compaction requested=false 2023-08-07 13:55:52,201 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/.tmp/info/2a6ffe12094b4b58a67fa9037378333e as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/info/2a6ffe12094b4b58a67fa9037378333e 2023-08-07 13:55:52,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/recovered.edits/109.seqid, newMaxSeqId=109, maxSeqId=1 2023-08-07 13:55:52,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:55:52,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e1cfded3452bdf9174e1f916e4d8e3f3: 2023-08-07 13:55:52,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:55:52,212 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing baa9fe21942fd12dea5caf4ddaa33efe, disabling compactions & flushes 2023-08-07 13:55:52,212 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:55:52,212 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:55:52,212 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. after waiting 0 ms 2023-08-07 13:55:52,212 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:55:52,212 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing baa9fe21942fd12dea5caf4ddaa33efe 2/2 column families, dataSize=12.70 KB heapSize=42.80 KB 2023-08-07 13:55:52,212 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/info/2a6ffe12094b4b58a67fa9037378333e, entries=30, sequenceid=19, filesize=8.1 K 2023-08-07 13:55:52,215 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/.tmp/rep_barrier/73c4a69ba56644f092aded864ae1f0b9 as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/rep_barrier/73c4a69ba56644f092aded864ae1f0b9 2023-08-07 13:55:52,218 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:55:52,220 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-08-07 13:55:52,220 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-07 13:55:52,220 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-08-07 13:55:52,225 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/rep_barrier/73c4a69ba56644f092aded864ae1f0b9, entries=2, sequenceid=19, filesize=4.9 K 2023-08-07 13:55:52,226 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-08-07 13:55:52,226 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-07 13:55:52,226 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-08-07 13:55:52,227 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/.tmp/table/1381c4f6b52d44928376e105bf7815d4 as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/table/1381c4f6b52d44928376e105bf7815d4 2023-08-07 13:55:52,248 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.74 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/.tmp/cf1/17cd2797e57d428da452894ee4464bca 2023-08-07 13:55:52,250 DEBUG [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(1504): Waiting on 1588230740, baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:55:52,251 INFO [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46383,1691416540182; all regions closed. 2023-08-07 13:55:52,251 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/table/1381c4f6b52d44928376e105bf7815d4, entries=6, sequenceid=19, filesize=4.8 K 2023-08-07 13:55:52,252 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.87 KB/3964, heapSize ~7.39 KB/7568, currentSize=0 B/0 for 1588230740 in 201ms, sequenceid=19, compaction requested=false 2023-08-07 13:55:52,264 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 17cd2797e57d428da452894ee4464bca 2023-08-07 13:55:52,267 DEBUG [RS:1;jenkins-hbase4:46383] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs 2023-08-07 13:55:52,268 INFO [RS:1;jenkins-hbase4:46383] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0:(num 1691416541682) 2023-08-07 13:55:52,276 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-08-07 13:55:52,277 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 13:55:52,278 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 13:55:52,278 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 13:55:52,279 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-07 13:55:52,281 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-07 13:55:52,281 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-07 13:55:52,281 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-08-07 13:55:52,282 DEBUG [RS:1;jenkins-hbase4:46383] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs 2023-08-07 13:55:52,282 INFO [RS:1;jenkins-hbase4:46383] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1:(num 1691416542713) 2023-08-07 13:55:52,282 DEBUG [RS:1;jenkins-hbase4:46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:52,282 INFO [RS:1;jenkins-hbase4:46383] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:55:52,282 INFO [RS:1;jenkins-hbase4:46383] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-07 13:55:52,283 INFO [RS:1;jenkins-hbase4:46383] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 13:55:52,283 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 13:55:52,283 INFO [RS:1;jenkins-hbase4:46383] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 13:55:52,283 INFO [RS:1;jenkins-hbase4:46383] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 13:55:52,283 INFO [RS:1;jenkins-hbase4:46383] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-07 13:55:52,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.96 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/.tmp/norep/f7d8ee36c40b4ad5b67c550fef4a0b08 2023-08-07 13:55:52,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f7d8ee36c40b4ad5b67c550fef4a0b08 2023-08-07 13:55:52,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/.tmp/cf1/17cd2797e57d428da452894ee4464bca as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/cf1/17cd2797e57d428da452894ee4464bca 2023-08-07 13:55:52,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 17cd2797e57d428da452894ee4464bca 2023-08-07 13:55:52,304 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/cf1/17cd2797e57d428da452894ee4464bca, entries=200, sequenceid=206, filesize=11.9 K 2023-08-07 13:55:52,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/.tmp/norep/f7d8ee36c40b4ad5b67c550fef4a0b08 as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/norep/f7d8ee36c40b4ad5b67c550fef4a0b08 2023-08-07 13:55:52,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f7d8ee36c40b4ad5b67c550fef4a0b08 2023-08-07 13:55:52,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/norep/f7d8ee36c40b4ad5b67c550fef4a0b08, entries=101, sequenceid=206, filesize=8.3 K 2023-08-07 13:55:52,312 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~12.70 KB/13001, heapSize ~42.77 KB/43800, currentSize=0 B/0 for baa9fe21942fd12dea5caf4ddaa33efe in 100ms, sequenceid=206, compaction requested=false 2023-08-07 13:55:52,319 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0.1691416541682 was moved to hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0.1691416541682 2023-08-07 13:55:52,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/recovered.edits/209.seqid, newMaxSeqId=209, maxSeqId=1 2023-08-07 13:55:52,322 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:55:52,322 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for baa9fe21942fd12dea5caf4ddaa33efe: 2023-08-07 13:55:52,322 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1.1691416542713 was moved to hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1.1691416542713 2023-08-07 13:55:52,322 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:55:52,330 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-08-07 13:55:52,387 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-08-07 13:55:52,387 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-07 13:55:52,434 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a10017, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:52,434 DEBUG [RS:1;jenkins-hbase4:46383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3dea9ca3 to 127.0.0.1:52488 2023-08-07 13:55:52,437 DEBUG [RS:1;jenkins-hbase4:46383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:52,438 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-08-07 13:55:52,450 INFO [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38873,1691416540136; all regions closed. 2023-08-07 13:55:52,454 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.meta.1691416541913.meta not finished, retry = 0 2023-08-07 13:55:52,523 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-07 13:55:52,523 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-07 13:55:52,537 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:381) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) 2023-08-07 13:55:52,537 INFO [RS:1;jenkins-hbase4:46383] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1 terminated 2023-08-07 13:55:52,539 INFO [RS:1;jenkins-hbase4:46383] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1 terminated 2023-08-07 13:55:52,540 INFO [RS:1;jenkins-hbase4:46383] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46383 2023-08-07 13:55:52,542 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-07 13:55:52,542 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:46383-0x101dfb6a1a10002, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:55:52,542 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:55:52,542 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:46383-0x101dfb6a1a10002, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-07 13:55:52,542 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-07 13:55:52,543 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46383,1691416540182] 2023-08-07 13:55:52,544 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46383,1691416540182; numProcessing=1 2023-08-07 13:55:52,545 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-08-07 13:55:52,546 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,46383,1691416540182 already deleted, retry=false 2023-08-07 13:55:52,546 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46383,1691416540182 expired; onlineServers=1 2023-08-07 13:55:52,557 DEBUG [RS:0;jenkins-hbase4:38873] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs 2023-08-07 13:55:52,557 INFO [RS:0;jenkins-hbase4:38873] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38873%2C1691416540136.meta:.meta(num 1691416541913) 2023-08-07 13:55:52,562 DEBUG [RS:0;jenkins-hbase4:38873] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs 2023-08-07 13:55:52,562 INFO [RS:0;jenkins-hbase4:38873] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0:(num 1691416541672) 2023-08-07 13:55:52,567 DEBUG [RS:0;jenkins-hbase4:38873] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs 2023-08-07 13:55:52,568 INFO [RS:0;jenkins-hbase4:38873] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1:(num 1691416546583) 2023-08-07 13:55:52,568 DEBUG [RS:0;jenkins-hbase4:38873] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:52,568 INFO [RS:0;jenkins-hbase4:38873] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:55:52,568 INFO [RS:0;jenkins-hbase4:38873] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-07 13:55:52,568 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 13:55:52,568 INFO [RS:0;jenkins-hbase4:38873] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-07 13:55:52,644 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:46383-0x101dfb6a1a10002, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:52,644 INFO [RS:1;jenkins-hbase4:46383] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46383,1691416540182; zookeeper connection closed. 2023-08-07 13:55:52,644 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:46383-0x101dfb6a1a10002, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:52,644 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@56134aee] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@56134aee 2023-08-07 13:55:52,647 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1.1691416546583 was moved to hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1.1691416546583 2023-08-07 13:55:52,731 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 13:55:52,744 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a10018, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:52,744 DEBUG [RS:0;jenkins-hbase4:38873] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7eb6373d to 127.0.0.1:52488 2023-08-07 13:55:52,744 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a10018, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:52,744 DEBUG [RS:0;jenkins-hbase4:38873] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:52,845 INFO [RS:0;jenkins-hbase4:38873] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1 terminated 2023-08-07 13:55:52,845 INFO [RS:0;jenkins-hbase4:38873] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1 terminated 2023-08-07 13:55:52,847 INFO [RS:0;jenkins-hbase4:38873] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38873 2023-08-07 13:55:52,850 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:55:52,850 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-07 13:55:52,851 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38873,1691416540136] 2023-08-07 13:55:52,852 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38873,1691416540136; numProcessing=2 2023-08-07 13:55:52,853 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,38873,1691416540136 already deleted, retry=false 2023-08-07 13:55:52,853 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38873,1691416540136 expired; onlineServers=0 2023-08-07 13:55:52,853 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38803,1691416538961' ***** 2023-08-07 13:55:52,853 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-07 13:55:52,854 DEBUG [M:0;jenkins-hbase4:38803] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d663dfd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 13:55:52,854 INFO [M:0;jenkins-hbase4:38803] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38803,1691416538961 2023-08-07 13:55:52,854 INFO [M:0;jenkins-hbase4:38803] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38803,1691416538961; all regions closed. 2023-08-07 13:55:52,854 DEBUG [M:0;jenkins-hbase4:38803] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:52,854 DEBUG [M:0;jenkins-hbase4:38803] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-07 13:55:52,854 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-07 13:55:52,854 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416541351] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416541351,5,FailOnTimeoutGroup] 2023-08-07 13:55:52,854 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416541351] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416541351,5,FailOnTimeoutGroup] 2023-08-07 13:55:52,854 DEBUG [M:0;jenkins-hbase4:38803] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-07 13:55:52,856 INFO [M:0;jenkins-hbase4:38803] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-07 13:55:52,856 INFO [M:0;jenkins-hbase4:38803] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-07 13:55:52,856 INFO [M:0;jenkins-hbase4:38803] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-07 13:55:52,856 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-08-07 13:55:52,856 DEBUG [M:0;jenkins-hbase4:38803] master.HMaster(1512): Stopping service threads 2023-08-07 13:55:52,856 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:55:52,856 INFO [M:0;jenkins-hbase4:38803] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-07 13:55:52,857 ERROR [M:0;jenkins-hbase4:38803] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] 2023-08-07 13:55:52,857 INFO [M:0;jenkins-hbase4:38803] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 13:55:52,857 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-07 13:55:52,857 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-07 13:55:52,858 DEBUG [M:0;jenkins-hbase4:38803] zookeeper.ZKUtil(398): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-08-07 13:55:52,858 WARN [M:0;jenkins-hbase4:38803] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-07 13:55:52,858 INFO [M:0;jenkins-hbase4:38803] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 13:55:52,858 INFO [M:0;jenkins-hbase4:38803] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-07 13:55:52,859 DEBUG [M:0;jenkins-hbase4:38803] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 13:55:52,859 INFO [M:0;jenkins-hbase4:38803] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:52,859 DEBUG [M:0;jenkins-hbase4:38803] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:52,859 DEBUG [M:0;jenkins-hbase4:38803] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 13:55:52,859 DEBUG [M:0;jenkins-hbase4:38803] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:52,859 INFO [M:0;jenkins-hbase4:38803] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=61.04 KB heapSize=72.60 KB 2023-08-07 13:55:52,873 INFO [M:0;jenkins-hbase4:38803] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=61.04 KB at sequenceid=136 (bloomFilter=true), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/123f9fcac0f443918e3bb3b963aa39ac 2023-08-07 13:55:52,880 DEBUG [M:0;jenkins-hbase4:38803] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/123f9fcac0f443918e3bb3b963aa39ac as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/123f9fcac0f443918e3bb3b963aa39ac 2023-08-07 13:55:52,885 INFO [M:0;jenkins-hbase4:38803] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/123f9fcac0f443918e3bb3b963aa39ac, entries=17, sequenceid=136, filesize=8.5 K 2023-08-07 13:55:52,886 INFO [M:0;jenkins-hbase4:38803] regionserver.HRegion(2948): Finished flush of dataSize ~61.04 KB/62508, heapSize ~72.59 KB/74328, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 27ms, sequenceid=136, compaction requested=false 2023-08-07 13:55:52,888 INFO [M:0;jenkins-hbase4:38803] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:55:52,888 DEBUG [M:0;jenkins-hbase4:38803] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 13:55:52,893 INFO [M:0;jenkins-hbase4:38803] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-07 13:55:52,893 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 13:55:52,894 INFO [M:0;jenkins-hbase4:38803] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38803 2023-08-07 13:55:52,896 DEBUG [M:0;jenkins-hbase4:38803] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,38803,1691416538961 already deleted, retry=false 2023-08-07 13:55:52,952 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:52,952 INFO [RS:0;jenkins-hbase4:38873] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38873,1691416540136; zookeeper connection closed. 2023-08-07 13:55:52,952 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): regionserver:38873-0x101dfb6a1a10001, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:52,952 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@22e829cd] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@22e829cd 2023-08-07 13:55:52,952 INFO [Listener at localhost/41247] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 2 regionserver(s) complete 2023-08-07 13:55:53,052 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:53,052 INFO [M:0;jenkins-hbase4:38803] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38803,1691416538961; zookeeper connection closed. 2023-08-07 13:55:53,052 DEBUG [Listener at localhost/39609-EventThread] zookeeper.ZKWatcher(600): master:38803-0x101dfb6a1a10000, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:55:53,055 INFO [Listener at localhost/41247] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-08-07 13:55:53,055 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:53,055 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:53,055 INFO [Listener at localhost/41247] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 13:55:53,055 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:53,055 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 13:55:53,056 INFO [Listener at localhost/41247] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 13:55:53,056 INFO [Listener at localhost/41247] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43899 2023-08-07 13:55:53,057 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:53,058 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:53,059 INFO [Listener at localhost/41247] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43899 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:53,063 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:438990x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:53,063 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:43899-0x101dfb6a1a1001c connected 2023-08-07 13:55:53,066 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-07 13:55:53,066 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:55:53,067 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-07 13:55:53,069 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43899 2023-08-07 13:55:53,069 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43899 2023-08-07 13:55:53,069 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43899 2023-08-07 13:55:53,070 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43899 2023-08-07 13:55:53,070 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43899 2023-08-07 13:55:53,070 INFO [Listener at localhost/41247] master.HMaster(444): hbase.rootdir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2, hbase.cluster.distributed=false 2023-08-07 13:55:53,071 DEBUG [pool-460-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-08-07 13:55:53,083 INFO [Listener at localhost/41247] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-07 13:55:53,083 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:53,083 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:53,083 INFO [Listener at localhost/41247] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 13:55:53,083 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:55:53,083 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 13:55:53,083 INFO [Listener at localhost/41247] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 13:55:53,084 INFO [Listener at localhost/41247] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46513 2023-08-07 13:55:53,084 INFO [Listener at localhost/41247] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 13:55:53,086 DEBUG [Listener at localhost/41247] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 13:55:53,086 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:53,087 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:53,088 INFO [Listener at localhost/41247] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46513 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:53,092 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:465130x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:53,094 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46513-0x101dfb6a1a1001d connected 2023-08-07 13:55:53,094 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:46513-0x101dfb6a1a1001d, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-07 13:55:53,095 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:46513-0x101dfb6a1a1001d, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:55:53,095 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:46513-0x101dfb6a1a1001d, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-07 13:55:53,095 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46513 2023-08-07 13:55:53,095 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46513 2023-08-07 13:55:53,096 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46513 2023-08-07 13:55:53,096 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46513 2023-08-07 13:55:53,096 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46513 2023-08-07 13:55:53,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,43899,1691416553054 2023-08-07 13:55:53,100 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-07 13:55:53,100 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,43899,1691416553054 2023-08-07 13:55:53,102 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-07 13:55:53,102 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46513-0x101dfb6a1a1001d, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-07 13:55:53,102 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:55:53,103 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-07 13:55:53,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,43899,1691416553054 from backup master directory 2023-08-07 13:55:53,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-07 13:55:53,106 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,43899,1691416553054 2023-08-07 13:55:53,106 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 13:55:53,106 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-07 13:55:53,107 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,43899,1691416553054 2023-08-07 13:55:53,121 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:55:53,149 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x3ca3d84d to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:53,155 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5c2b6e8d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:53,155 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 13:55:53,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-07 13:55:53,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:55:53,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:55:53,164 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,45527,1691416544159 to hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,45527,1691416544159-dead as it is dead 2023-08-07 13:55:53,165 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,45527,1691416544159-dead/jenkins-hbase4.apache.org%2C45527%2C1691416544159.jenkins-hbase4.apache.org%2C45527%2C1691416544159.regiongroup-0.1691416544600 2023-08-07 13:55:53,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,45527,1691416544159-dead/jenkins-hbase4.apache.org%2C45527%2C1691416544159.jenkins-hbase4.apache.org%2C45527%2C1691416544159.regiongroup-0.1691416544600 after 5ms 2023-08-07 13:55:53,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,45527,1691416544159-dead/jenkins-hbase4.apache.org%2C45527%2C1691416544159.jenkins-hbase4.apache.org%2C45527%2C1691416544159.regiongroup-0.1691416544600 to hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C45527%2C1691416544159.jenkins-hbase4.apache.org%2C45527%2C1691416544159.regiongroup-0.1691416544600 2023-08-07 13:55:53,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,45527,1691416544159-dead 2023-08-07 13:55:53,171 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,43899,1691416553054 2023-08-07 13:55:53,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:53,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43899%2C1691416553054.jenkins-hbase4.apache.org%2C43899%2C1691416553054.regiongroup-0, suffix=, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,43899,1691416553054, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/oldWALs, maxLogs=10 2023-08-07 13:55:53,189 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:55:53,190 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK] 2023-08-07 13:55:53,191 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK] 2023-08-07 13:55:53,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,43899,1691416553054/jenkins-hbase4.apache.org%2C43899%2C1691416553054.jenkins-hbase4.apache.org%2C43899%2C1691416553054.regiongroup-0.1691416553174 2023-08-07 13:55:53,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK], DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK], DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK]] 2023-08-07 13:55:53,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:53,194 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:53,194 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:53,194 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:53,195 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:53,196 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-07 13:55:53,197 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-07 13:55:53,203 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5eec9c061c4641719002e9ed31c10ef2 2023-08-07 13:55:53,204 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:53,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-08-07 13:55:53,205 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C45527%2C1691416544159.jenkins-hbase4.apache.org%2C45527%2C1691416544159.regiongroup-0.1691416544600 2023-08-07 13:55:53,211 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 131, firstSequenceIdInLog=3, maxSequenceIdInLog=118, path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C45527%2C1691416544159.jenkins-hbase4.apache.org%2C45527%2C1691416544159.regiongroup-0.1691416544600 2023-08-07 13:55:53,212 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C45527%2C1691416544159.jenkins-hbase4.apache.org%2C45527%2C1691416544159.regiongroup-0.1691416544600 2023-08-07 13:55:53,214 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:55:53,217 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=1 2023-08-07 13:55:53,218 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=119; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71997547, jitterRate=0.07284705340862274}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 13:55:53,218 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 13:55:53,218 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-07 13:55:53,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-07 13:55:53,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-07 13:55:53,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 13:55:53,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 13:55:53,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 13:55:53,235 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 13:55:53,235 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 13:55:53,236 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 13:55:53,236 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-08-07 13:55:53,237 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-08-07 13:55:53,237 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 16 msec 2023-08-07 13:55:53,237 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-07 13:55:53,238 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 13:55:53,239 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46589,1691416544393, table=hbase:meta, region=1588230740 2023-08-07 13:55:53,241 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 4 possibly 'live' servers, and 0 'splitting'. 2023-08-07 13:55:53,243 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,35577,1691416544304 already deleted, retry=false 2023-08-07 13:55:53,243 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,35577,1691416544304 on jenkins-hbase4.apache.org,43899,1691416553054 2023-08-07 13:55:53,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,35577,1691416544304, splitWal=true, meta=false 2023-08-07 13:55:53,252 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=15 for jenkins-hbase4.apache.org,35577,1691416544304 (carryingMeta=false) jenkins-hbase4.apache.org,35577,1691416544304/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@76a58702[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-07 13:55:53,254 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,34709,1691416544248 already deleted, retry=false 2023-08-07 13:55:53,254 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,34709,1691416544248 on jenkins-hbase4.apache.org,43899,1691416553054 2023-08-07 13:55:53,255 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=16, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,34709,1691416544248, splitWal=true, meta=false 2023-08-07 13:55:53,255 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=16 for jenkins-hbase4.apache.org,34709,1691416544248 (carryingMeta=false) jenkins-hbase4.apache.org,34709,1691416544248/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@435c8159[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-07 13:55:53,256 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,41049,1691416544342 already deleted, retry=false 2023-08-07 13:55:53,256 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,41049,1691416544342 on jenkins-hbase4.apache.org,43899,1691416553054 2023-08-07 13:55:53,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,41049,1691416544342, splitWal=true, meta=false 2023-08-07 13:55:53,257 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=17 for jenkins-hbase4.apache.org,41049,1691416544342 (carryingMeta=false) jenkins-hbase4.apache.org,41049,1691416544342/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@191b83a7[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-07 13:55:53,259 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46589,1691416544393 already deleted, retry=false 2023-08-07 13:55:53,259 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,46589,1691416544393 on jenkins-hbase4.apache.org,43899,1691416553054 2023-08-07 13:55:53,259 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,46589,1691416544393, splitWal=true, meta=true 2023-08-07 13:55:53,259 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,46589,1691416544393 (carryingMeta=true) jenkins-hbase4.apache.org,46589,1691416544393/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@278e0a36[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-07 13:55:53,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-07 13:55:53,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-07 13:55:53,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-08-07 13:55:53,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-07 13:55:53,271 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-08-07 13:55:53,272 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-08-07 13:55:53,272 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-08-07 13:55:53,273 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-08-07 13:55:53,274 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46513-0x101dfb6a1a1001d, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-07 13:55:53,274 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-07 13:55:53,275 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:55:53,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,43899,1691416553054, sessionid=0x101dfb6a1a1001c, setting cluster-up flag (Was=false) 2023-08-07 13:55:53,279 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-08-07 13:55:53,280 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43899,1691416553054 2023-08-07 13:55:53,283 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-08-07 13:55:53,284 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43899,1691416553054 2023-08-07 13:55:53,285 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.hbase-snapshot/.tmp 2023-08-07 13:55:53,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:55:53,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:55:53,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:55:53,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:55:53,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-07 13:55:53,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:53,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 13:55:53,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:53,291 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691416583290 2023-08-07 13:55:53,292 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-07 13:55:53,292 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-07 13:55:53,292 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-07 13:55:53,292 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-07 13:55:53,292 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-07 13:55:53,292 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-07 13:55:53,293 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:53,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-07 13:55:53,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-07 13:55:53,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-07 13:55:53,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-07 13:55:53,295 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-07 13:55:53,297 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46589,1691416544393; numProcessing=1 2023-08-07 13:55:53,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416553295,5,FailOnTimeoutGroup] 2023-08-07 13:55:53,298 DEBUG [PEWorker-5] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41049,1691416544342; numProcessing=2 2023-08-07 13:55:53,298 DEBUG [PEWorker-4] master.DeadServer(103): Processing jenkins-hbase4.apache.org,35577,1691416544304; numProcessing=3 2023-08-07 13:55:53,298 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416553298,5,FailOnTimeoutGroup] 2023-08-07 13:55:53,298 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34709,1691416544248; numProcessing=4 2023-08-07 13:55:53,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:53,298 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=16, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34709,1691416544248, splitWal=true, meta=false 2023-08-07 13:55:53,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-07 13:55:53,298 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46589,1691416544393, splitWal=true, meta=true 2023-08-07 13:55:53,298 INFO [PEWorker-5] procedure.ServerCrashProcedure(161): Start pid=17, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,41049,1691416544342, splitWal=true, meta=false 2023-08-07 13:55:53,298 INFO [PEWorker-4] procedure.ServerCrashProcedure(161): Start pid=15, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,35577,1691416544304, splitWal=true, meta=false 2023-08-07 13:55:53,299 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:53,299 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:53,299 INFO [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(951): ClusterId : bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:55:53,299 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691416553299, completionTime=-1 2023-08-07 13:55:53,299 DEBUG [RS:0;jenkins-hbase4:46513] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 13:55:53,299 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-08-07 13:55:53,299 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-07 13:55:53,300 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46589,1691416544393, splitWal=true, meta=true, isMeta: true 2023-08-07 13:55:53,302 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46589,1691416544393-splitting 2023-08-07 13:55:53,303 DEBUG [RS:0;jenkins-hbase4:46513] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 13:55:53,303 DEBUG [RS:0;jenkins-hbase4:46513] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 13:55:53,303 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46589,1691416544393-splitting dir is empty, no logs to split. 2023-08-07 13:55:53,303 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46589,1691416544393 WAL count=0, meta=true 2023-08-07 13:55:53,308 DEBUG [RS:0;jenkins-hbase4:46513] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 13:55:53,309 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46589,1691416544393-splitting dir is empty, no logs to split. 2023-08-07 13:55:53,310 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46589,1691416544393 WAL count=0, meta=true 2023-08-07 13:55:53,310 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46589,1691416544393 WAL splitting is done? wals=0, meta=true 2023-08-07 13:55:53,310 DEBUG [RS:0;jenkins-hbase4:46513] zookeeper.ReadOnlyZKClient(139): Connect 0x3b13b2a5 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:53,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-07 13:55:53,315 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-07 13:55:53,317 DEBUG [RS:0;jenkins-hbase4:46513] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@35d44bb7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:53,317 DEBUG [RS:0;jenkins-hbase4:46513] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@31ebd9d5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 13:55:53,317 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-07 13:55:53,327 DEBUG [RS:0;jenkins-hbase4:46513] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:46513 2023-08-07 13:55:53,327 INFO [RS:0;jenkins-hbase4:46513] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 13:55:53,327 INFO [RS:0;jenkins-hbase4:46513] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 13:55:53,327 DEBUG [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 13:55:53,328 INFO [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,43899,1691416553054 with isa=jenkins-hbase4.apache.org/172.31.14.131:46513, startcode=1691416553082 2023-08-07 13:55:53,328 DEBUG [RS:0;jenkins-hbase4:46513] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 13:55:53,330 INFO [RS-EventLoopGroup-11-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55107, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 13:55:53,331 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43899] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:55:53,332 DEBUG [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2 2023-08-07 13:55:53,332 DEBUG [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45993 2023-08-07 13:55:53,332 DEBUG [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 13:55:53,334 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:55:53,334 DEBUG [RS:0;jenkins-hbase4:46513] zookeeper.ZKUtil(162): regionserver:46513-0x101dfb6a1a1001d, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:55:53,334 WARN [RS:0;jenkins-hbase4:46513] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 13:55:53,335 INFO [RS:0;jenkins-hbase4:46513] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:55:53,336 INFO [RS:0;jenkins-hbase4:46513] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:55:53,336 DEBUG [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:55:53,336 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46513,1691416553082] 2023-08-07 13:55:53,340 DEBUG [RS:0;jenkins-hbase4:46513] zookeeper.ZKUtil(162): regionserver:46513-0x101dfb6a1a1001d, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:55:53,341 DEBUG [RS:0;jenkins-hbase4:46513] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-07 13:55:53,341 INFO [RS:0;jenkins-hbase4:46513] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 13:55:53,343 INFO [RS:0;jenkins-hbase4:46513] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 13:55:53,344 INFO [RS:0;jenkins-hbase4:46513] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 13:55:53,344 INFO [RS:0;jenkins-hbase4:46513] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:53,344 INFO [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 13:55:53,346 INFO [RS:0;jenkins-hbase4:46513] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:53,347 DEBUG [RS:0;jenkins-hbase4:46513] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:53,347 DEBUG [RS:0;jenkins-hbase4:46513] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:53,347 DEBUG [RS:0;jenkins-hbase4:46513] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:53,347 DEBUG [RS:0;jenkins-hbase4:46513] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:53,347 DEBUG [RS:0;jenkins-hbase4:46513] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:53,348 DEBUG [RS:0;jenkins-hbase4:46513] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 13:55:53,348 DEBUG [RS:0;jenkins-hbase4:46513] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:53,348 DEBUG [RS:0;jenkins-hbase4:46513] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:53,348 DEBUG [RS:0;jenkins-hbase4:46513] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:53,348 DEBUG [RS:0;jenkins-hbase4:46513] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:55:53,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-07 13:55:53,353 INFO [RS:0;jenkins-hbase4:46513] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:53,353 INFO [RS:0;jenkins-hbase4:46513] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:53,353 INFO [RS:0;jenkins-hbase4:46513] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:53,368 INFO [RS:0;jenkins-hbase4:46513] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 13:55:53,368 INFO [RS:0;jenkins-hbase4:46513] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46513,1691416553082-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:53,379 INFO [RS:0;jenkins-hbase4:46513] regionserver.Replication(203): jenkins-hbase4.apache.org,46513,1691416553082 started 2023-08-07 13:55:53,379 INFO [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46513,1691416553082, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46513, sessionid=0x101dfb6a1a1001d 2023-08-07 13:55:53,379 DEBUG [RS:0;jenkins-hbase4:46513] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 13:55:53,379 DEBUG [RS:0;jenkins-hbase4:46513] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:55:53,379 DEBUG [RS:0;jenkins-hbase4:46513] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46513,1691416553082' 2023-08-07 13:55:53,379 DEBUG [RS:0;jenkins-hbase4:46513] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-07 13:55:53,380 DEBUG [RS:0;jenkins-hbase4:46513] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-07 13:55:53,380 DEBUG [RS:0;jenkins-hbase4:46513] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 13:55:53,380 DEBUG [RS:0;jenkins-hbase4:46513] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 13:55:53,380 DEBUG [RS:0;jenkins-hbase4:46513] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:55:53,380 DEBUG [RS:0;jenkins-hbase4:46513] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46513,1691416553082' 2023-08-07 13:55:53,380 DEBUG [RS:0;jenkins-hbase4:46513] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-07 13:55:53,381 DEBUG [RS:0;jenkins-hbase4:46513] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-07 13:55:53,381 DEBUG [RS:0;jenkins-hbase4:46513] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 13:55:53,381 INFO [RS:0;jenkins-hbase4:46513] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 13:55:53,381 INFO [RS:0;jenkins-hbase4:46513] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 13:55:53,467 DEBUG [jenkins-hbase4:43899] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-07 13:55:53,469 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46513,1691416553082, state=OPENING 2023-08-07 13:55:53,470 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-07 13:55:53,470 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-07 13:55:53,470 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46513,1691416553082}] 2023-08-07 13:55:53,481 INFO [RS:0;jenkins-hbase4:46513] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:53,483 INFO [RS:0;jenkins-hbase4:46513] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46513%2C1691416553082.jenkins-hbase4.apache.org%2C46513%2C1691416553082.regiongroup-0, suffix=, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46513,1691416553082, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs, maxLogs=10 2023-08-07 13:55:53,499 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK] 2023-08-07 13:55:53,499 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:55:53,499 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK] 2023-08-07 13:55:53,502 INFO [RS:0;jenkins-hbase4:46513] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46513,1691416553082/jenkins-hbase4.apache.org%2C46513%2C1691416553082.jenkins-hbase4.apache.org%2C46513%2C1691416553082.regiongroup-0.1691416553484 2023-08-07 13:55:53,502 DEBUG [RS:0;jenkins-hbase4:46513] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK], DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK], DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK]] 2023-08-07 13:55:53,625 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:55:53,626 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 13:55:53,627 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55516, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 13:55:53,631 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-07 13:55:53,631 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:55:53,631 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:55:53,631 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:53,633 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46513%2C1691416553082.meta, suffix=.meta, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46513,1691416553082, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs, maxLogs=10 2023-08-07 13:55:53,647 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:55:53,647 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK] 2023-08-07 13:55:53,647 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK] 2023-08-07 13:55:53,649 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46513,1691416553082/jenkins-hbase4.apache.org%2C46513%2C1691416553082.meta.1691416553633.meta 2023-08-07 13:55:53,649 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK], DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK], DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK]] 2023-08-07 13:55:53,649 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:53,649 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-07 13:55:53,649 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-07 13:55:53,651 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-07 13:55:53,651 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-07 13:55:53,651 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:53,651 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-07 13:55:53,651 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-07 13:55:53,652 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 13:55:53,653 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info 2023-08-07 13:55:53,654 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info 2023-08-07 13:55:53,654 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 13:55:53,661 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info/8766d5bf2c17410e8580146469a8662e 2023-08-07 13:55:53,661 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:53,661 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 13:55:53,662 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/rep_barrier 2023-08-07 13:55:53,662 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/rep_barrier 2023-08-07 13:55:53,663 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 13:55:53,663 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:53,663 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 13:55:53,664 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/table 2023-08-07 13:55:53,664 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/table 2023-08-07 13:55:53,665 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 13:55:53,671 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/table/b6b6cd7163ac41e2b3c4c475d90c98bf 2023-08-07 13:55:53,671 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:53,672 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740 2023-08-07 13:55:53,673 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740 2023-08-07 13:55:53,676 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-07 13:55:53,678 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 13:55:53,679 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70206859, jitterRate=0.04616372287273407}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-07 13:55:53,679 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 13:55:53,680 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=20, masterSystemTime=1691416553625 2023-08-07 13:55:53,685 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-07 13:55:53,685 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-07 13:55:53,686 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46513,1691416553082, state=OPEN 2023-08-07 13:55:53,688 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-07 13:55:53,689 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-07 13:55:53,691 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-08-07 13:55:53,691 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46513,1691416553082 in 219 msec 2023-08-07 13:55:53,693 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=18 2023-08-07 13:55:53,693 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 379 msec 2023-08-07 13:55:54,853 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1554ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1504ms 2023-08-07 13:55:56,355 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3056ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3006ms 2023-08-07 13:55:57,669 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 13:55:57,808 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4509ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-08-07 13:55:57,808 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 13:55:57,812 DEBUG [hconnection-0x49414d06-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:57,813 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55522, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:55:57,823 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=3bc256268a4ab1eec2fd5f34b09fb147, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46589,1691416544393, regionLocation=jenkins-hbase4.apache.org,46589,1691416544393, openSeqNum=2 2023-08-07 13:55:57,823 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=77564e07be12a17717602214bc11ceb0, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35577,1691416544304, regionLocation=jenkins-hbase4.apache.org,35577,1691416544304, openSeqNum=2 2023-08-07 13:55:57,823 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=75110bd10fe307a72d13190ea044da7a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34709,1691416544248, regionLocation=jenkins-hbase4.apache.org,34709,1691416544248, openSeqNum=2 2023-08-07 13:55:57,824 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-07 13:55:57,824 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691416617824 2023-08-07 13:55:57,824 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691416677824 2023-08-07 13:55:57,824 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 15 msec 2023-08-07 13:55:57,849 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43899,1691416553054-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:57,849 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43899,1691416553054-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:57,849 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43899,1691416553054-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:57,849 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:43899, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:57,849 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-07 13:55:57,850 INFO [PEWorker-1] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,41049,1691416544342 had 0 regions 2023-08-07 13:55:57,850 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,46589,1691416544393 had 2 regions 2023-08-07 13:55:57,850 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,35577,1691416544304 had 1 regions 2023-08-07 13:55:57,850 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,34709,1691416544248 had 1 regions 2023-08-07 13:55:57,850 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. is NOT online; state={3bc256268a4ab1eec2fd5f34b09fb147 state=OPEN, ts=1691416557823, server=jenkins-hbase4.apache.org,46589,1691416544393}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-08-07 13:55:57,851 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=17, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,41049,1691416544342, splitWal=true, meta=false, isMeta: false 2023-08-07 13:55:57,851 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46589,1691416544393, splitWal=true, meta=true, isMeta: false 2023-08-07 13:55:57,851 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34709,1691416544248, splitWal=true, meta=false, isMeta: false 2023-08-07 13:55:57,851 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,35577,1691416544304, splitWal=true, meta=false, isMeta: false 2023-08-07 13:55:57,853 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,41049,1691416544342-splitting 2023-08-07 13:55:57,854 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,41049,1691416544342-splitting dir is empty, no logs to split. 2023-08-07 13:55:57,854 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,41049,1691416544342 WAL count=0, meta=false 2023-08-07 13:55:57,855 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46589,1691416544393-splitting dir is empty, no logs to split. 2023-08-07 13:55:57,855 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,46589,1691416544393 WAL count=0, meta=false 2023-08-07 13:55:57,856 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,46589,1691416544393/hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147., unknown_server=jenkins-hbase4.apache.org,35577,1691416544304/t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0., unknown_server=jenkins-hbase4.apache.org,34709,1691416544248/t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:57,856 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,34709,1691416544248-splitting 2023-08-07 13:55:57,857 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,34709,1691416544248-splitting dir is empty, no logs to split. 2023-08-07 13:55:57,857 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,34709,1691416544248 WAL count=0, meta=false 2023-08-07 13:55:57,857 DEBUG [PEWorker-2] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,35577,1691416544304-splitting 2023-08-07 13:55:57,858 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,35577,1691416544304-splitting dir is empty, no logs to split. 2023-08-07 13:55:57,858 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,35577,1691416544304 WAL count=0, meta=false 2023-08-07 13:55:57,859 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,41049,1691416544342-splitting dir is empty, no logs to split. 2023-08-07 13:55:57,859 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,41049,1691416544342 WAL count=0, meta=false 2023-08-07 13:55:57,859 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,41049,1691416544342 WAL splitting is done? wals=0, meta=false 2023-08-07 13:55:57,860 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46589,1691416544393-splitting dir is empty, no logs to split. 2023-08-07 13:55:57,860 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,46589,1691416544393 WAL count=0, meta=false 2023-08-07 13:55:57,860 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46589,1691416544393 WAL splitting is done? wals=0, meta=false 2023-08-07 13:55:57,862 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,34709,1691416544248-splitting dir is empty, no logs to split. 2023-08-07 13:55:57,862 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,34709,1691416544248 WAL count=0, meta=false 2023-08-07 13:55:57,862 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34709,1691416544248 WAL splitting is done? wals=0, meta=false 2023-08-07 13:55:57,862 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=3bc256268a4ab1eec2fd5f34b09fb147, ASSIGN}] 2023-08-07 13:55:57,862 INFO [PEWorker-1] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,41049,1691416544342 failed, ignore...File hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,41049,1691416544342-splitting does not exist. 2023-08-07 13:55:57,864 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=3bc256268a4ab1eec2fd5f34b09fb147, ASSIGN 2023-08-07 13:55:57,864 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,35577,1691416544304-splitting dir is empty, no logs to split. 2023-08-07 13:55:57,864 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,35577,1691416544304 WAL count=0, meta=false 2023-08-07 13:55:57,864 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,35577,1691416544304 WAL splitting is done? wals=0, meta=false 2023-08-07 13:55:57,865 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,41049,1691416544342 after splitting done 2023-08-07 13:55:57,865 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,41049,1691416544342 from processing; numProcessing=3 2023-08-07 13:55:57,865 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,34709,1691416544248 failed, ignore...File hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,34709,1691416544248-splitting does not exist. 2023-08-07 13:55:57,865 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=21, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=3bc256268a4ab1eec2fd5f34b09fb147, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-07 13:55:57,866 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=75110bd10fe307a72d13190ea044da7a, ASSIGN}] 2023-08-07 13:55:57,867 INFO [PEWorker-2] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,35577,1691416544304 failed, ignore...File hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,35577,1691416544304-splitting does not exist. 2023-08-07 13:55:57,867 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,41049,1691416544342, splitWal=true, meta=false in 4.6090 sec 2023-08-07 13:55:57,868 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=75110bd10fe307a72d13190ea044da7a, ASSIGN 2023-08-07 13:55:57,868 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=77564e07be12a17717602214bc11ceb0, ASSIGN}] 2023-08-07 13:55:57,869 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=75110bd10fe307a72d13190ea044da7a, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-07 13:55:57,869 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=77564e07be12a17717602214bc11ceb0, ASSIGN 2023-08-07 13:55:57,870 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=77564e07be12a17717602214bc11ceb0, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-07 13:55:58,016 DEBUG [jenkins-hbase4:43899] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-07 13:55:58,018 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=3bc256268a4ab1eec2fd5f34b09fb147, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:55:58,018 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=75110bd10fe307a72d13190ea044da7a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:55:58,018 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=77564e07be12a17717602214bc11ceb0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:55:58,019 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416558018"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416558018"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416558018"}]},"ts":"1691416558018"} 2023-08-07 13:55:58,019 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691416558018"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416558018"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416558018"}]},"ts":"1691416558018"} 2023-08-07 13:55:58,019 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416558018"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416558018"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416558018"}]},"ts":"1691416558018"} 2023-08-07 13:55:58,021 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE; OpenRegionProcedure 77564e07be12a17717602214bc11ceb0, server=jenkins-hbase4.apache.org,46513,1691416553082}] 2023-08-07 13:55:58,022 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure 75110bd10fe307a72d13190ea044da7a, server=jenkins-hbase4.apache.org,46513,1691416553082}] 2023-08-07 13:55:58,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=21, state=RUNNABLE; OpenRegionProcedure 3bc256268a4ab1eec2fd5f34b09fb147, server=jenkins-hbase4.apache.org,46513,1691416553082}] 2023-08-07 13:55:58,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:58,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:55:58,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46513%2C1691416553082.jenkins-hbase4.apache.org%2C46513%2C1691416553082.regiongroup-1, suffix=, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46513,1691416553082, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs, maxLogs=10 2023-08-07 13:55:58,182 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:58,183 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3bc256268a4ab1eec2fd5f34b09fb147, NAME => 'hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:58,183 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:58,183 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:58,183 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:58,183 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:58,185 INFO [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:58,186 DEBUG [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/info 2023-08-07 13:55:58,187 DEBUG [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/info 2023-08-07 13:55:58,187 INFO [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bc256268a4ab1eec2fd5f34b09fb147 columnFamilyName info 2023-08-07 13:55:58,194 DEBUG [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/info/2db3ce6e71eb4d9a95061ddf2a1afa69 2023-08-07 13:55:58,194 INFO [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] regionserver.HStore(310): Store=3bc256268a4ab1eec2fd5f34b09fb147/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:58,195 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:58,196 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:58,197 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK] 2023-08-07 13:55:58,197 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:55:58,197 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK] 2023-08-07 13:55:58,201 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:55:58,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46513,1691416553082/jenkins-hbase4.apache.org%2C46513%2C1691416553082.jenkins-hbase4.apache.org%2C46513%2C1691416553082.regiongroup-1.1691416558182 2023-08-07 13:55:58,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK], DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK], DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK]] 2023-08-07 13:55:58,205 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3bc256268a4ab1eec2fd5f34b09fb147; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74891634, jitterRate=0.11597231030464172}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 13:55:58,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 77564e07be12a17717602214bc11ceb0, NAME => 't1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:58,206 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3bc256268a4ab1eec2fd5f34b09fb147: 2023-08-07 13:55:58,206 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:58,206 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:58,206 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:58,206 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:58,209 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:58,209 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147., pid=26, masterSystemTime=1691416558174 2023-08-07 13:55:58,210 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1 2023-08-07 13:55:58,210 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1 2023-08-07 13:55:58,210 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 77564e07be12a17717602214bc11ceb0 columnFamilyName cf1 2023-08-07 13:55:58,212 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:58,213 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:55:58,214 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=3bc256268a4ab1eec2fd5f34b09fb147, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:55:58,214 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691416558213"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416558213"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416558213"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416558213"}]},"ts":"1691416558213"} 2023-08-07 13:55:58,217 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=21 2023-08-07 13:55:58,217 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=21, state=SUCCESS; OpenRegionProcedure 3bc256268a4ab1eec2fd5f34b09fb147, server=jenkins-hbase4.apache.org,46513,1691416553082 in 192 msec 2023-08-07 13:55:58,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=18 2023-08-07 13:55:58,220 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,46589,1691416544393 after splitting done 2023-08-07 13:55:58,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=3bc256268a4ab1eec2fd5f34b09fb147, ASSIGN in 355 msec 2023-08-07 13:55:58,220 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,46589,1691416544393 from processing; numProcessing=2 2023-08-07 13:55:58,223 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,46589,1691416544393, splitWal=true, meta=true in 4.9610 sec 2023-08-07 13:55:58,223 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1/ed63a5fa2177464491100ded6dbb022b 2023-08-07 13:55:58,224 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(310): Store=77564e07be12a17717602214bc11ceb0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:58,224 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:58,225 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/norep 2023-08-07 13:55:58,225 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/norep 2023-08-07 13:55:58,225 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 77564e07be12a17717602214bc11ceb0 columnFamilyName norep 2023-08-07 13:55:58,226 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(310): Store=77564e07be12a17717602214bc11ceb0/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:58,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:58,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:58,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 13:55:58,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 77564e07be12a17717602214bc11ceb0 2023-08-07 13:55:58,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 77564e07be12a17717602214bc11ceb0; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63714907, jitterRate=-0.05057390034198761}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 13:55:58,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 77564e07be12a17717602214bc11ceb0: 2023-08-07 13:55:58,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0., pid=24, masterSystemTime=1691416558174 2023-08-07 13:55:58,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:58,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:55:58,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:58,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 75110bd10fe307a72d13190ea044da7a, NAME => 't2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:55:58,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:58,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:55:58,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:58,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:58,237 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=77564e07be12a17717602214bc11ceb0, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:55:58,237 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416558237"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416558237"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416558237"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416558237"}]},"ts":"1691416558237"} 2023-08-07 13:55:58,238 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:58,239 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1 2023-08-07 13:55:58,239 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1 2023-08-07 13:55:58,240 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75110bd10fe307a72d13190ea044da7a columnFamilyName cf1 2023-08-07 13:55:58,241 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-08-07 13:55:58,241 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; OpenRegionProcedure 77564e07be12a17717602214bc11ceb0, server=jenkins-hbase4.apache.org,46513,1691416553082 in 218 msec 2023-08-07 13:55:58,244 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=15 2023-08-07 13:55:58,244 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=77564e07be12a17717602214bc11ceb0, ASSIGN in 373 msec 2023-08-07 13:55:58,244 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,35577,1691416544304 after splitting done 2023-08-07 13:55:58,244 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,35577,1691416544304 from processing; numProcessing=1 2023-08-07 13:55:58,245 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,35577,1691416544304, splitWal=true, meta=false in 5.0000 sec 2023-08-07 13:55:58,247 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1/8a858ad05b094550b24538261ebbea4e 2023-08-07 13:55:58,247 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(310): Store=75110bd10fe307a72d13190ea044da7a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:58,247 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:58,248 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/norep 2023-08-07 13:55:58,249 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/norep 2023-08-07 13:55:58,249 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75110bd10fe307a72d13190ea044da7a columnFamilyName norep 2023-08-07 13:55:58,249 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(310): Store=75110bd10fe307a72d13190ea044da7a/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:55:58,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:58,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:58,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 13:55:58,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:55:58,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 75110bd10fe307a72d13190ea044da7a; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58910721, jitterRate=-0.1221618503332138}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 13:55:58,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 75110bd10fe307a72d13190ea044da7a: 2023-08-07 13:55:58,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a., pid=25, masterSystemTime=1691416558174 2023-08-07 13:55:58,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:58,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:55:58,270 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=75110bd10fe307a72d13190ea044da7a, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:55:58,270 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416558270"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416558270"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416558270"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416558270"}]},"ts":"1691416558270"} 2023-08-07 13:55:58,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-08-07 13:55:58,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure 75110bd10fe307a72d13190ea044da7a, server=jenkins-hbase4.apache.org,46513,1691416553082 in 250 msec 2023-08-07 13:55:58,279 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=16 2023-08-07 13:55:58,279 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,34709,1691416544248 after splitting done 2023-08-07 13:55:58,279 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=16, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=75110bd10fe307a72d13190ea044da7a, ASSIGN in 407 msec 2023-08-07 13:55:58,279 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,34709,1691416544248 from processing; numProcessing=0 2023-08-07 13:55:58,282 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34709,1691416544248, splitWal=true, meta=false in 5.0250 sec 2023-08-07 13:55:58,379 INFO [jenkins-hbase4:46513Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:55:58,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-08-07 13:55:58,866 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-08-07 13:55:58,868 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-08-07 13:55:58,869 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.761sec 2023-08-07 13:55:58,869 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-07 13:55:58,869 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-07 13:55:58,869 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-07 13:55:58,869 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43899,1691416553054-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-07 13:55:58,869 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43899,1691416553054-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-07 13:55:58,872 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-07 13:55:58,905 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(139): Connect 0x31acef0d to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:58,912 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@68f139b2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:58,914 DEBUG [hconnection-0x4527bc6a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:58,916 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55532, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:55:58,920 INFO [Listener at localhost/41247] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-08-07 13:55:58,921 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x31acef0d to 127.0.0.1:52488 2023-08-07 13:55:58,921 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:55:58,922 INFO [Listener at localhost/41247] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:43899 after: jenkins-hbase4.apache.org:43899 2023-08-07 13:55:58,922 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(139): Connect 0x08fe00ad to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:58,927 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@70422934, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:59,341 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-08-07 13:55:59,342 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-07 13:55:59,342 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-07 13:55:59,343 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-08-07 13:55:59,429 DEBUG [hconnection-0x41b5ddfc-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:55:59,431 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55542, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:55:59,453 INFO [Listener at localhost/41247] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1691416559453 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:59,456 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): syncupReplication16914165594530x0, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:59,457 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1691416559453-0x101dfb6a1a10022 connected Start Replication Server start 2023-08-07 13:55:59,460 INFO [Listener at localhost/41247] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:55:59,460 INFO [Listener at localhost/41247] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:55:59,472 INFO [Listener at localhost/41247] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-07 13:55:59,473 DEBUG [Listener at localhost/41247] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-07 13:55:59,483 INFO [Listener at localhost/41247] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-07 13:55:59,485 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,38873,1691416540136, jenkins-hbase4.apache.org,46383,1691416540182] other RSs: [] 2023-08-07 13:55:59,510 DEBUG [Listener at localhost/41247.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x25020784 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:55:59,516 DEBUG [Listener at localhost/41247.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@5e42bf9f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:55:59,516 INFO [Listener at localhost/41247.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:55:59,520 DEBUG [Listener at localhost/41247.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:55:59,521 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101dfb6a1a10024 connected 2023-08-07 13:55:59,521 INFO [Listener at localhost/41247.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=4d10afbc-00b5-4a8d-bcc6-ae0771b33a8c to cluster=bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:56:02,939 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,38873,1691416540136/1's WALs to 1691416559460.syncuptool.replication.org,1234,1 2023-08-07 13:56:02,941 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1.1691416546583 with data PBUF\x08\xDFU 2023-08-07 13:56:02,942 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0.1691416541672 with data PBUF\x08\xF8\xA9\x01 2023-08-07 13:56:02,944 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,38873,1691416540136/1's WALs to 1691416559460.syncuptool.replication.org,1234,1 2023-08-07 13:56:03,379 INFO [jenkins-hbase4:46513Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:56:03,687 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 13:56:04,947 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,38873,1691416540136] 2023-08-07 13:56:04,947 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,38873,1691416540136] 2023-08-07 13:56:04,963 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,38873,1691416540136] 2023-08-07 13:56:04,963 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,38873,1691416540136, ReplicationSource: 1, currentBandwidth=0 2023-08-07 13:56:04,963 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:56:04,984 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136] zookeeper.ReadOnlyZKClient(139): Connect 0x3258a3be to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:04,990 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@211a2fb7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:04,991 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:56:05,000 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:56:05,001 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101dfb6a1a10026 connected 2023-08-07 13:56:05,001 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,38873,1691416540136 (queues=2) is replicating from cluster=4d10afbc-00b5-4a8d-bcc6-ae0771b33a8c to cluster=bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:56:05,001 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1 2023-08-07 13:56:05,008 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,38873,1691416540136, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-07 13:56:05,008 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0 2023-08-07 13:56:05,008 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1 2023-08-07 13:56:05,010 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,38873,1691416540136, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-07 13:56:05,011 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0 2023-08-07 13:56:05,029 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1.1691416546583 2023-08-07 13:56:05,030 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-08-07 13:56:05,030 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-07 13:56:05,031 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0.1691416541672 2023-08-07 13:56:05,031 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,38873,1691416540136 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1.1691416546583, lastWalPosition=15173, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=true,usedBufferSize=4120] 2023-08-07 13:56:05,031 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-08-07 13:56:05,032 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-07 13:56:05,035 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,38873,1691416540136 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs/jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0.1691416541672, lastWalPosition=29258, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=true,usedBufferSize=8120] 2023-08-07 13:56:05,036 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 13:56:05,037 INFO [SinkThread-1] regionserver.ReplicationSinkManager(110): Current list of sinks is out of date or empty, updating 2023-08-07 13:56:05,042 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45830, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 13:56:05,043 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46513] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-07 13:56:05,044 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=46513] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-07 13:56:05,044 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46513] zookeeper.ReadOnlyZKClient(139): Connect 0x3c5b0eb3 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:05,050 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46513] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@329ce748, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:05,053 DEBUG [hconnection-0x36a365c3-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:56:05,055 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45842, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=ClientService 2023-08-07 13:56:05,067 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46513] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-07 13:56:05,067 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1-jenkins-hbase4.apache.org,38873,1691416540136] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,38873,1691416540136] 2023-08-07 13:56:05,070 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=46513] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-07 13:56:05,071 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1-jenkins-hbase4.apache.org,38873,1691416540136] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,38873,1691416540136] 2023-08-07 13:56:05,073 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1.1691416546583] 2023-08-07 13:56:05,073 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0.1691416541672] 2023-08-07 13:56:05,076 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,38873,1691416540136 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-07 13:56:05,076 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:56:05,076 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,38873,1691416540136 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-07 13:56:05,076 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:56:05,176 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:56:05,179 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691416559453-0x101dfb6a1a10022, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1691416559460.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:56:05,179 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-1,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,38873,1691416540136 with the following stats: Total replicated edits: 2, current progress: 2023-08-07 13:56:05,280 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,38873,1691416540136 2023-08-07 13:56:05,280 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1-jenkins-hbase4.apache.org,38873,1691416540136] zookeeper.ZKUtil(243): syncupReplication1691416559453-0x101dfb6a1a10022, quorum=127.0.0.1:52488, baseZNode=/1 Unable to list children of znode /1/replication/rs/1691416559460.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,38873,1691416540136 because node does not exist (not an error) 2023-08-07 13:56:05,281 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136.replicationSource.shipperjenkins-hbase4.apache.org%2C38873%2C1691416540136.jenkins-hbase4.apache.org%2C38873%2C1691416540136.regiongroup-0,1-jenkins-hbase4.apache.org,38873,1691416540136] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,38873,1691416540136 with the following stats: Total replicated edits: 2, current progress: 2023-08-07 13:56:08,379 INFO [jenkins-hbase4:46513Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:56:08,379 INFO [jenkins-hbase4:46513Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-08-07 13:56:08,624 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,46383,1691416540182/1's WALs to 1691416559460.syncuptool.replication.org,1234,1 2023-08-07 13:56:08,626 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1.1691416542713 with data PBUF\x08\xCD\x06 2023-08-07 13:56:08,627 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0.1691416541682 with data 2023-08-07 13:56:08,629 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,46383,1691416540182/1's WALs to 1691416559460.syncuptool.replication.org,1234,1 2023-08-07 13:56:10,630 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,46383,1691416540182] 2023-08-07 13:56:10,631 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,46383,1691416540182] 2023-08-07 13:56:10,641 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,46383,1691416540182] 2023-08-07 13:56:10,642 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,46383,1691416540182, ReplicationSource: 1, currentBandwidth=0 2023-08-07 13:56:10,642 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:56:10,662 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182] zookeeper.ReadOnlyZKClient(139): Connect 0x713b6b67 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:10,669 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@35eb2082, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:10,669 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:56:10,672 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:56:10,674 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101dfb6a1a10029 connected 2023-08-07 13:56:10,674 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,46383,1691416540182 (queues=2) is replicating from cluster=4d10afbc-00b5-4a8d-bcc6-ae0771b33a8c to cluster=bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:56:10,674 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1 2023-08-07 13:56:10,675 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,46383,1691416540182, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-07 13:56:10,679 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 13:56:10,682 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0 2023-08-07 13:56:10,682 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1 2023-08-07 13:56:10,686 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,46383,1691416540182, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-07 13:56:10,692 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0 2023-08-07 13:56:10,698 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1.1691416542713 2023-08-07 13:56:10,698 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-07 13:56:10,698 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-07 13:56:10,698 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,46383,1691416540182 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1.1691416542713, lastWalPosition=1430, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-08-07 13:56:10,699 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1-jenkins-hbase4.apache.org,46383,1691416540182] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,46383,1691416540182] 2023-08-07 13:56:10,701 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1.1691416542713] 2023-08-07 13:56:10,703 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0.1691416541682 2023-08-07 13:56:10,703 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-07 13:56:10,703 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,46383,1691416540182 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-07 13:56:10,704 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:56:10,707 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,46383,1691416540182 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-07 13:56:10,707 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:56:10,805 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:56:10,806 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1-jenkins-hbase4.apache.org,46383,1691416540182] zookeeper.ZKUtil(162): syncupReplication1691416559453-0x101dfb6a1a10022, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/replication/rs/1691416559460.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,46383,1691416540182/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0.1691416541682 2023-08-07 13:56:10,813 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691416559453-0x101dfb6a1a10022, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1691416559460.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,46383,1691416540182/jenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0.1691416541682 2023-08-07 13:56:10,813 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-0,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,46383,1691416540182 with the following stats: Total replicated edits: 0, current progress: 2023-08-07 13:56:10,813 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691416559453-0x101dfb6a1a10022, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/replication/rs/1691416559460.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:56:10,815 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691416559453-0x101dfb6a1a10022, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1691416559460.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:56:10,914 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,46383,1691416540182 2023-08-07 13:56:10,914 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1-jenkins-hbase4.apache.org,46383,1691416540182] zookeeper.ZKUtil(243): syncupReplication1691416559453-0x101dfb6a1a10022, quorum=127.0.0.1:52488, baseZNode=/1 Unable to list children of znode /1/replication/rs/1691416559460.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,46383,1691416540182 because node does not exist (not an error) 2023-08-07 13:56:10,915 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182.replicationSource.shipperjenkins-hbase4.apache.org%2C46383%2C1691416540182.jenkins-hbase4.apache.org%2C46383%2C1691416540182.regiongroup-1,1-jenkins-hbase4.apache.org,46383,1691416540182] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,46383,1691416540182 with the following stats: Total replicated edits: 0, current progress: 2023-08-07 13:56:13,379 INFO [jenkins-hbase4:46513Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:56:13,379 INFO [jenkins-hbase4:46513Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-08-07 13:56:16,308 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 13:56:18,379 INFO [jenkins-hbase4:46513Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:56:18,379 INFO [jenkins-hbase4:46513Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-08-07 13:56:19,486 INFO [Listener at localhost/41247] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-07 13:56:19,591 DEBUG [Listener at localhost/41247.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a10024, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:19,591 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x25020784 to 127.0.0.1:52488 2023-08-07 13:56:19,592 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:19,795 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691416559453-0x101dfb6a1a10022, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:19,796 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691416559453-0x101dfb6a1a10022, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:19,814 INFO [Listener at localhost/41247] replication.TestReplicationSyncUpTool(203): SyncUpAfterDelete succeeded at retry = 0 2023-08-07 13:56:19,815 DEBUG [Listener at localhost/41247] replication.TestReplicationSyncUpTool(215): mimicSyncUpAfterPut 2023-08-07 13:56:19,817 INFO [Listener at localhost/41247] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 13:56:19,817 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:56:19,817 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 13:56:19,817 INFO [Listener at localhost/41247] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 13:56:19,817 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:56:19,817 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 13:56:19,818 INFO [Listener at localhost/41247] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 13:56:19,819 INFO [Listener at localhost/41247] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44307 2023-08-07 13:56:19,819 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:56:19,821 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:56:19,822 INFO [Listener at localhost/41247] zookeeper.RecoverableZooKeeper(93): Process identifier=master:44307 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:56:19,825 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:443070x0, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:56:19,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:44307-0x101dfb6a1a1002a connected 2023-08-07 13:56:19,829 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-07 13:56:19,829 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-07 13:56:19,830 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-07 13:56:19,835 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44307 2023-08-07 13:56:19,835 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44307 2023-08-07 13:56:19,837 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44307 2023-08-07 13:56:19,838 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44307 2023-08-07 13:56:19,839 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44307 2023-08-07 13:56:19,839 INFO [Listener at localhost/41247] master.HMaster(444): hbase.rootdir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a, hbase.cluster.distributed=false 2023-08-07 13:56:19,842 DEBUG [pool-564-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: INIT 2023-08-07 13:56:19,860 INFO [Listener at localhost/41247] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-07 13:56:19,860 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:56:19,860 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 13:56:19,860 INFO [Listener at localhost/41247] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 13:56:19,860 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:56:19,861 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 13:56:19,861 INFO [Listener at localhost/41247] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 13:56:19,862 INFO [Listener at localhost/41247] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34615 2023-08-07 13:56:19,862 INFO [Listener at localhost/41247] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 13:56:19,870 DEBUG [Listener at localhost/41247] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 13:56:19,871 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:56:19,873 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:56:19,874 INFO [Listener at localhost/41247] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34615 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:56:19,876 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:346150x0, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:56:19,879 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:346150x0, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-07 13:56:19,879 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34615-0x101dfb6a1a1002b connected 2023-08-07 13:56:19,880 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:34615-0x101dfb6a1a1002b, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-07 13:56:19,880 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:34615-0x101dfb6a1a1002b, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-08-07 13:56:19,883 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34615 2023-08-07 13:56:19,883 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34615 2023-08-07 13:56:19,887 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34615 2023-08-07 13:56:19,889 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34615 2023-08-07 13:56:19,889 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34615 2023-08-07 13:56:19,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,44307,1691416579816 2023-08-07 13:56:19,897 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-08-07 13:56:19,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,44307,1691416579816 2023-08-07 13:56:19,902 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-07 13:56:19,902 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34615-0x101dfb6a1a1002b, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-08-07 13:56:19,902 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:56:19,905 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/master 2023-08-07 13:56:19,905 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/master 2023-08-07 13:56:19,905 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,44307,1691416579816 from backup master directory 2023-08-07 13:56:19,908 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,44307,1691416579816 2023-08-07 13:56:19,908 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-08-07 13:56:19,908 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 13:56:19,908 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,44307,1691416579816 2023-08-07 13:56:19,929 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:56:20,363 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x5da4278d to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:20,366 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@24244511, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:20,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 13:56:20,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-07 13:56:20,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:56:20,369 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:56:20,375 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/WALs/jenkins-hbase4.apache.org,38803,1691416538961 to hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/WALs/jenkins-hbase4.apache.org,38803,1691416538961-dead as it is dead 2023-08-07 13:56:20,375 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/WALs/jenkins-hbase4.apache.org,38803,1691416538961-dead/jenkins-hbase4.apache.org%2C38803%2C1691416538961.jenkins-hbase4.apache.org%2C38803%2C1691416538961.regiongroup-0.1691416540619 2023-08-07 13:56:20,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/WALs/jenkins-hbase4.apache.org,38803,1691416538961-dead/jenkins-hbase4.apache.org%2C38803%2C1691416538961.jenkins-hbase4.apache.org%2C38803%2C1691416538961.regiongroup-0.1691416540619 after 0ms 2023-08-07 13:56:20,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/WALs/jenkins-hbase4.apache.org,38803,1691416538961-dead/jenkins-hbase4.apache.org%2C38803%2C1691416538961.jenkins-hbase4.apache.org%2C38803%2C1691416538961.regiongroup-0.1691416540619 to hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38803%2C1691416538961.jenkins-hbase4.apache.org%2C38803%2C1691416538961.regiongroup-0.1691416540619 2023-08-07 13:56:20,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/WALs/jenkins-hbase4.apache.org,38803,1691416538961-dead 2023-08-07 13:56:20,378 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/WALs/jenkins-hbase4.apache.org,44307,1691416579816 2023-08-07 13:56:20,379 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:56:20,380 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44307%2C1691416579816.jenkins-hbase4.apache.org%2C44307%2C1691416579816.regiongroup-0, suffix=, logDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/WALs/jenkins-hbase4.apache.org,44307,1691416579816, archiveDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/oldWALs, maxLogs=10 2023-08-07 13:56:20,395 DEBUG [RS-EventLoopGroup-14-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK] 2023-08-07 13:56:20,397 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK] 2023-08-07 13:56:20,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/WALs/jenkins-hbase4.apache.org,44307,1691416579816/jenkins-hbase4.apache.org%2C44307%2C1691416579816.jenkins-hbase4.apache.org%2C44307%2C1691416579816.regiongroup-0.1691416580381 2023-08-07 13:56:20,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK], DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK]] 2023-08-07 13:56:20,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:56:20,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:56:20,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:56:20,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:56:20,401 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:56:20,403 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-07 13:56:20,403 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-07 13:56:20,415 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/123f9fcac0f443918e3bb3b963aa39ac 2023-08-07 13:56:20,415 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:20,415 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-08-07 13:56:20,416 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38803%2C1691416538961.jenkins-hbase4.apache.org%2C38803%2C1691416538961.regiongroup-0.1691416540619 2023-08-07 13:56:20,420 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 155, firstSequenceIdInLog=3, maxSequenceIdInLog=138, path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38803%2C1691416538961.jenkins-hbase4.apache.org%2C38803%2C1691416538961.regiongroup-0.1691416540619 2023-08-07 13:56:20,421 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C38803%2C1691416538961.jenkins-hbase4.apache.org%2C38803%2C1691416538961.regiongroup-0.1691416540619 2023-08-07 13:56:20,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:56:20,427 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/138.seqid, newMaxSeqId=138, maxSeqId=1 2023-08-07 13:56:20,428 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=139; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71351640, jitterRate=0.06322228908538818}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 13:56:20,428 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 13:56:20,428 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-07 13:56:20,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-07 13:56:20,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-07 13:56:20,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 13:56:20,431 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 13:56:20,435 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 13:56:20,436 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 13:56:20,436 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 13:56:20,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 13:56:20,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-08-07 13:56:20,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-08-07 13:56:20,438 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-08-07 13:56:20,438 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-07 13:56:20,438 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-07 13:56:20,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 13:56:20,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,38873,1691416540136, table=hbase:meta, region=1588230740 2023-08-07 13:56:20,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 2 possibly 'live' servers, and 0 'splitting'. 2023-08-07 13:56:20,444 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,38873,1691416540136 already deleted, retry=false 2023-08-07 13:56:20,445 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,38873,1691416540136 on jenkins-hbase4.apache.org,44307,1691416579816 2023-08-07 13:56:20,446 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,38873,1691416540136, splitWal=true, meta=true 2023-08-07 13:56:20,446 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,38873,1691416540136 (carryingMeta=true) jenkins-hbase4.apache.org,38873,1691416540136/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@51dc3d8b[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-07 13:56:20,447 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,46383,1691416540182 already deleted, retry=false 2023-08-07 13:56:20,447 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,46383,1691416540182 on jenkins-hbase4.apache.org,44307,1691416579816 2023-08-07 13:56:20,448 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,46383,1691416540182, splitWal=true, meta=false 2023-08-07 13:56:20,448 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=19 for jenkins-hbase4.apache.org,46383,1691416540182 (carryingMeta=false) jenkins-hbase4.apache.org,46383,1691416540182/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@3965fe9c[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-07 13:56:20,458 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-07 13:56:20,458 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-07 13:56:20,458 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-08-07 13:56:20,459 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-07 13:56:20,459 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-08-07 13:56:20,460 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-08-07 13:56:20,460 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-08-07 13:56:20,462 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-08-07 13:56:20,463 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-07 13:56:20,463 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34615-0x101dfb6a1a1002b, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-08-07 13:56:20,463 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:56:20,467 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,44307,1691416579816, sessionid=0x101dfb6a1a1002a, setting cluster-up flag (Was=false) 2023-08-07 13:56:20,471 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-08-07 13:56:20,472 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,44307,1691416579816 2023-08-07 13:56:20,474 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-08-07 13:56:20,474 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,44307,1691416579816 2023-08-07 13:56:20,475 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/.hbase-snapshot/.tmp 2023-08-07 13:56:20,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:56:20,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:56:20,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:56:20,477 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:56:20,477 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-07 13:56:20,477 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:20,477 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 13:56:20,477 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:20,479 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691416610478 2023-08-07 13:56:20,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-07 13:56:20,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-07 13:56:20,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-07 13:56:20,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-07 13:56:20,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-07 13:56:20,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-07 13:56:20,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:20,482 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46383,1691416540182; numProcessing=1 2023-08-07 13:56:20,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-07 13:56:20,482 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=19, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46383,1691416540182, splitWal=true, meta=false 2023-08-07 13:56:20,482 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38873,1691416540136; numProcessing=2 2023-08-07 13:56:20,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-07 13:56:20,482 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38873,1691416540136, splitWal=true, meta=true 2023-08-07 13:56:20,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-07 13:56:20,483 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38873,1691416540136, splitWal=true, meta=true, isMeta: true 2023-08-07 13:56:20,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-07 13:56:20,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-07 13:56:20,486 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416580486,5,FailOnTimeoutGroup] 2023-08-07 13:56:20,486 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416580486,5,FailOnTimeoutGroup] 2023-08-07 13:56:20,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:20,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-07 13:56:20,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:20,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:20,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691416580486, completionTime=-1 2023-08-07 13:56:20,486 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-08-07 13:56:20,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-07 13:56:20,487 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136-splitting 2023-08-07 13:56:20,488 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136-splitting dir is empty, no logs to split. 2023-08-07 13:56:20,488 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,38873,1691416540136 WAL count=0, meta=true 2023-08-07 13:56:20,490 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136-splitting dir is empty, no logs to split. 2023-08-07 13:56:20,490 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,38873,1691416540136 WAL count=0, meta=true 2023-08-07 13:56:20,491 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,38873,1691416540136 WAL splitting is done? wals=0, meta=true 2023-08-07 13:56:20,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-07 13:56:20,494 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-07 13:56:20,496 INFO [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(951): ClusterId : 4d10afbc-00b5-4a8d-bcc6-ae0771b33a8c 2023-08-07 13:56:20,496 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-07 13:56:20,496 DEBUG [RS:0;jenkins-hbase4:34615] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 13:56:20,498 DEBUG [RS:0;jenkins-hbase4:34615] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 13:56:20,498 DEBUG [RS:0;jenkins-hbase4:34615] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 13:56:20,500 DEBUG [RS:0;jenkins-hbase4:34615] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 13:56:20,502 DEBUG [RS:0;jenkins-hbase4:34615] zookeeper.ReadOnlyZKClient(139): Connect 0x6b81ca89 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:20,508 DEBUG [RS:0;jenkins-hbase4:34615] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65462989, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:20,508 DEBUG [RS:0;jenkins-hbase4:34615] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6179a1f9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 13:56:20,521 DEBUG [RS:0;jenkins-hbase4:34615] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:34615 2023-08-07 13:56:20,521 INFO [RS:0;jenkins-hbase4:34615] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 13:56:20,521 INFO [RS:0;jenkins-hbase4:34615] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 13:56:20,521 DEBUG [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 13:56:20,522 INFO [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,44307,1691416579816 with isa=jenkins-hbase4.apache.org/172.31.14.131:34615, startcode=1691416579859 2023-08-07 13:56:20,522 DEBUG [RS:0;jenkins-hbase4:34615] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 13:56:20,524 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56197, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 13:56:20,526 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44307] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:20,526 DEBUG [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a 2023-08-07 13:56:20,527 DEBUG [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39335 2023-08-07 13:56:20,527 DEBUG [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 13:56:20,528 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-07 13:56:20,528 DEBUG [RS:0;jenkins-hbase4:34615] zookeeper.ZKUtil(162): regionserver:34615-0x101dfb6a1a1002b, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:20,529 WARN [RS:0;jenkins-hbase4:34615] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 13:56:20,529 INFO [RS:0;jenkins-hbase4:34615] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:56:20,530 INFO [RS:0;jenkins-hbase4:34615] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:56:20,530 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34615,1691416579859] 2023-08-07 13:56:20,531 DEBUG [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:20,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=51ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-07 13:56:20,552 INFO [RS:0;jenkins-hbase4:34615] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-07 13:56:20,553 DEBUG [RS:0;jenkins-hbase4:34615] zookeeper.ZKUtil(162): regionserver:34615-0x101dfb6a1a1002b, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:20,554 DEBUG [RS:0;jenkins-hbase4:34615] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-07 13:56:20,555 INFO [RS:0;jenkins-hbase4:34615] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 13:56:20,556 INFO [RS:0;jenkins-hbase4:34615] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 13:56:20,557 INFO [RS:0;jenkins-hbase4:34615] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 13:56:20,557 INFO [RS:0;jenkins-hbase4:34615] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:20,557 INFO [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 13:56:20,559 INFO [RS:0;jenkins-hbase4:34615] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:20,560 DEBUG [RS:0;jenkins-hbase4:34615] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:20,561 DEBUG [RS:0;jenkins-hbase4:34615] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:20,561 DEBUG [RS:0;jenkins-hbase4:34615] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:20,561 DEBUG [RS:0;jenkins-hbase4:34615] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:20,561 DEBUG [RS:0;jenkins-hbase4:34615] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:20,561 DEBUG [RS:0;jenkins-hbase4:34615] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 13:56:20,561 DEBUG [RS:0;jenkins-hbase4:34615] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:20,561 DEBUG [RS:0;jenkins-hbase4:34615] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:20,561 DEBUG [RS:0;jenkins-hbase4:34615] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:20,561 DEBUG [RS:0;jenkins-hbase4:34615] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:20,562 INFO [RS:0;jenkins-hbase4:34615] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:20,562 INFO [RS:0;jenkins-hbase4:34615] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:20,562 INFO [RS:0;jenkins-hbase4:34615] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:20,578 INFO [RS:0;jenkins-hbase4:34615] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 13:56:20,578 INFO [RS:0;jenkins-hbase4:34615] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34615,1691416579859-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:20,591 INFO [RS:0;jenkins-hbase4:34615] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-07 13:56:20,592 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [1691416559460.syncuptool.replication.org,1234,1] other RSs: [jenkins-hbase4.apache.org,34615,1691416579859] 2023-08-07 13:56:20,603 INFO [RS:0;jenkins-hbase4:34615] regionserver.Replication(203): jenkins-hbase4.apache.org,34615,1691416579859 started 2023-08-07 13:56:20,603 INFO [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34615,1691416579859, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34615, sessionid=0x101dfb6a1a1002b 2023-08-07 13:56:20,603 DEBUG [RS:0;jenkins-hbase4:34615] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 13:56:20,604 DEBUG [RS:0;jenkins-hbase4:34615] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:20,604 DEBUG [RS:0;jenkins-hbase4:34615] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34615,1691416579859' 2023-08-07 13:56:20,604 DEBUG [RS:0;jenkins-hbase4:34615] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-08-07 13:56:20,604 DEBUG [RS:0;jenkins-hbase4:34615] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-08-07 13:56:20,604 DEBUG [RS:0;jenkins-hbase4:34615] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 13:56:20,605 DEBUG [RS:0;jenkins-hbase4:34615] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 13:56:20,605 DEBUG [RS:0;jenkins-hbase4:34615] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:20,605 DEBUG [RS:0;jenkins-hbase4:34615] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34615,1691416579859' 2023-08-07 13:56:20,605 DEBUG [RS:0;jenkins-hbase4:34615] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-08-07 13:56:20,605 DEBUG [RS:0;jenkins-hbase4:34615] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-08-07 13:56:20,605 DEBUG [RS:0;jenkins-hbase4:34615] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 13:56:20,605 INFO [RS:0;jenkins-hbase4:34615] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 13:56:20,606 INFO [RS:0;jenkins-hbase4:34615] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 13:56:20,615 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x1e287bc8 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:20,620 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@117ddd2c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:20,620 INFO [RS:0;jenkins-hbase4:34615.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:56:20,622 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:56:20,625 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101dfb6a1a1002f connected 2023-08-07 13:56:20,625 INFO [RS:0;jenkins-hbase4:34615.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=4d10afbc-00b5-4a8d-bcc6-ae0771b33a8c to cluster=bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:56:20,646 DEBUG [jenkins-hbase4:44307] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-07 13:56:20,648 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34615,1691416579859, state=OPENING 2023-08-07 13:56:20,650 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-08-07 13:56:20,650 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-08-07 13:56:20,650 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34615,1691416579859}] 2023-08-07 13:56:20,706 INFO [RS:0;jenkins-hbase4:34615] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:56:20,708 INFO [RS:0;jenkins-hbase4:34615] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0, suffix=, logDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,34615,1691416579859, archiveDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs, maxLogs=10 2023-08-07 13:56:20,739 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK] 2023-08-07 13:56:20,739 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK] 2023-08-07 13:56:20,748 DEBUG [RS:0;jenkins-hbase4:34615] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0 for peer 1 2023-08-07 13:56:20,748 INFO [RS:0;jenkins-hbase4:34615] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,34615,1691416579859/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0.1691416580708 2023-08-07 13:56:20,750 DEBUG [RS:0;jenkins-hbase4:34615] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0 2023-08-07 13:56:20,750 INFO [RS:0;jenkins-hbase4:34615] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-07 13:56:20,751 DEBUG [RS:0;jenkins-hbase4:34615] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK], DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK]] 2023-08-07 13:56:20,751 INFO [RS:0;jenkins-hbase4:34615.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0 2023-08-07 13:56:20,759 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:20,808 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:20,808 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 13:56:20,809 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37510, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 13:56:20,813 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-07 13:56:20,813 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:56:20,813 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:56:20,813 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:56:20,815 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34615%2C1691416579859.meta, suffix=.meta, logDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,34615,1691416579859, archiveDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs, maxLogs=10 2023-08-07 13:56:20,828 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK] 2023-08-07 13:56:20,828 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK] 2023-08-07 13:56:20,830 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,34615,1691416579859/jenkins-hbase4.apache.org%2C34615%2C1691416579859.meta.1691416580816.meta 2023-08-07 13:56:20,831 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK], DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK]] 2023-08-07 13:56:20,831 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:56:20,831 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-07 13:56:20,831 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-07 13:56:20,831 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-07 13:56:20,831 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-07 13:56:20,831 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:56:20,831 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-07 13:56:20,831 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-07 13:56:20,833 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 13:56:20,834 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/info 2023-08-07 13:56:20,834 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/info 2023-08-07 13:56:20,834 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 13:56:20,841 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/info/2a6ffe12094b4b58a67fa9037378333e 2023-08-07 13:56:20,841 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:20,841 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 13:56:20,842 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/rep_barrier 2023-08-07 13:56:20,842 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/rep_barrier 2023-08-07 13:56:20,843 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 13:56:20,851 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/rep_barrier/73c4a69ba56644f092aded864ae1f0b9 2023-08-07 13:56:20,851 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:20,851 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 13:56:20,852 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/table 2023-08-07 13:56:20,852 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/table 2023-08-07 13:56:20,853 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 13:56:20,859 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/table/1381c4f6b52d44928376e105bf7815d4 2023-08-07 13:56:20,860 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:20,861 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740 2023-08-07 13:56:20,862 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740 2023-08-07 13:56:20,864 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:20,865 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-07 13:56:20,867 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 13:56:20,868 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66969544, jitterRate=-0.0020760297775268555}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-07 13:56:20,868 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 13:56:20,869 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=21, masterSystemTime=1691416580808 2023-08-07 13:56:20,874 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-07 13:56:20,875 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-07 13:56:20,875 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34615,1691416579859, state=OPEN 2023-08-07 13:56:20,877 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-08-07 13:56:20,878 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-08-07 13:56:20,880 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-08-07 13:56:20,880 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34615,1691416579859 in 228 msec 2023-08-07 13:56:20,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=18 2023-08-07 13:56:20,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 389 msec 2023-08-07 13:56:20,969 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:21,075 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:21,181 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:21,287 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:21,393 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:21,498 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:21,603 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:21,709 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:21,814 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:21,919 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:22,025 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:22,040 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1554ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-08-07 13:56:22,130 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:22,238 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:22,344 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:22,449 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:22,554 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:22,659 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:22,764 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:22,868 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:22,974 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:23,079 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:23,184 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:23,290 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:23,379 INFO [jenkins-hbase4:46513Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:56:23,379 INFO [jenkins-hbase4:46513Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-08-07 13:56:23,396 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:23,502 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:23,543 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3057ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3006ms 2023-08-07 13:56:23,608 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:23,713 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:23,819 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:23,925 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:24,029 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:24,135 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:24,240 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:24,353 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:24,458 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:24,564 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:24,670 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:24,775 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:24,880 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:24,985 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:24,995 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4509ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-08-07 13:56:24,996 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 13:56:24,999 DEBUG [hconnection-0x1be379b2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:56:25,001 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37516, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:56:25,006 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=ee0eecdf41bc57a826e75999fe370d6d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46383,1691416540182, regionLocation=jenkins-hbase4.apache.org,46383,1691416540182, openSeqNum=2 2023-08-07 13:56:25,007 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=e1cfded3452bdf9174e1f916e4d8e3f3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38873,1691416540136, regionLocation=jenkins-hbase4.apache.org,38873,1691416540136, openSeqNum=2 2023-08-07 13:56:25,007 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=baa9fe21942fd12dea5caf4ddaa33efe, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38873,1691416540136, regionLocation=jenkins-hbase4.apache.org,38873,1691416540136, openSeqNum=2 2023-08-07 13:56:25,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-07 13:56:25,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691416645007 2023-08-07 13:56:25,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691416705007 2023-08-07 13:56:25,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 11 msec 2023-08-07 13:56:25,025 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,38873,1691416540136 had 3 regions 2023-08-07 13:56:25,025 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,46383,1691416540182 had 1 regions 2023-08-07 13:56:25,025 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44307,1691416579816-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:25,025 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44307,1691416579816-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:25,025 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44307,1691416579816-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:25,025 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:44307, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:25,025 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:25,025 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. is NOT online; state={ee0eecdf41bc57a826e75999fe370d6d state=OPEN, ts=1691416585006, server=jenkins-hbase4.apache.org,46383,1691416540182}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-08-07 13:56:25,027 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,38873,1691416540136, splitWal=true, meta=true, isMeta: false 2023-08-07 13:56:25,027 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46383,1691416540182, splitWal=true, meta=false, isMeta: false 2023-08-07 13:56:25,030 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136-splitting dir is empty, no logs to split. 2023-08-07 13:56:25,030 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,38873,1691416540136 WAL count=0, meta=false 2023-08-07 13:56:25,030 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,46383,1691416540182/hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d., unknown_server=jenkins-hbase4.apache.org,38873,1691416540136/t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3., unknown_server=jenkins-hbase4.apache.org,38873,1691416540136/t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:56:25,031 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182-splitting 2023-08-07 13:56:25,032 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182-splitting dir is empty, no logs to split. 2023-08-07 13:56:25,032 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,46383,1691416540182 WAL count=0, meta=false 2023-08-07 13:56:25,033 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,38873,1691416540136-splitting dir is empty, no logs to split. 2023-08-07 13:56:25,033 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,38873,1691416540136 WAL count=0, meta=false 2023-08-07 13:56:25,033 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,38873,1691416540136 WAL splitting is done? wals=0, meta=false 2023-08-07 13:56:25,034 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182-splitting dir is empty, no logs to split. 2023-08-07 13:56:25,035 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,46383,1691416540182 WAL count=0, meta=false 2023-08-07 13:56:25,035 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46383,1691416540182 WAL splitting is done? wals=0, meta=false 2023-08-07 13:56:25,035 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=e1cfded3452bdf9174e1f916e4d8e3f3, ASSIGN}, {pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=baa9fe21942fd12dea5caf4ddaa33efe, ASSIGN}] 2023-08-07 13:56:25,036 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=e1cfded3452bdf9174e1f916e4d8e3f3, ASSIGN 2023-08-07 13:56:25,036 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=baa9fe21942fd12dea5caf4ddaa33efe, ASSIGN 2023-08-07 13:56:25,036 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=e1cfded3452bdf9174e1f916e4d8e3f3, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-07 13:56:25,037 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=baa9fe21942fd12dea5caf4ddaa33efe, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-07 13:56:25,037 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,46383,1691416540182 failed, ignore...File hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,46383,1691416540182-splitting does not exist. 2023-08-07 13:56:25,039 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ee0eecdf41bc57a826e75999fe370d6d, ASSIGN}] 2023-08-07 13:56:25,040 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ee0eecdf41bc57a826e75999fe370d6d, ASSIGN 2023-08-07 13:56:25,040 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=ee0eecdf41bc57a826e75999fe370d6d, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-07 13:56:25,041 DEBUG [jenkins-hbase4:44307] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-07 13:56:25,044 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=baa9fe21942fd12dea5caf4ddaa33efe, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:25,044 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=e1cfded3452bdf9174e1f916e4d8e3f3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:25,044 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416585044"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416585044"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416585044"}]},"ts":"1691416585044"} 2023-08-07 13:56:25,044 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=ee0eecdf41bc57a826e75999fe370d6d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:25,044 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416585044"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416585044"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416585044"}]},"ts":"1691416585044"} 2023-08-07 13:56:25,044 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691416585044"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416585044"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416585044"}]},"ts":"1691416585044"} 2023-08-07 13:56:25,046 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=23, state=RUNNABLE; OpenRegionProcedure baa9fe21942fd12dea5caf4ddaa33efe, server=jenkins-hbase4.apache.org,34615,1691416579859}] 2023-08-07 13:56:25,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=22, state=RUNNABLE; OpenRegionProcedure e1cfded3452bdf9174e1f916e4d8e3f3, server=jenkins-hbase4.apache.org,34615,1691416579859}] 2023-08-07 13:56:25,048 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=24, state=RUNNABLE; OpenRegionProcedure ee0eecdf41bc57a826e75999fe370d6d, server=jenkins-hbase4.apache.org,34615,1691416579859}] 2023-08-07 13:56:25,091 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:25,098 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 13:56:25,208 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:25,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:56:25,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:56:25,221 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:56:25,221 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ee0eecdf41bc57a826e75999fe370d6d, NAME => 'hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:56:25,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1, suffix=, logDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,34615,1691416579859, archiveDir=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs, maxLogs=10 2023-08-07 13:56:25,222 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:56:25,222 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:56:25,222 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:56:25,222 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:56:25,229 INFO [StoreOpener-ee0eecdf41bc57a826e75999fe370d6d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:56:25,231 DEBUG [StoreOpener-ee0eecdf41bc57a826e75999fe370d6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d/info 2023-08-07 13:56:25,231 DEBUG [StoreOpener-ee0eecdf41bc57a826e75999fe370d6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d/info 2023-08-07 13:56:25,231 INFO [StoreOpener-ee0eecdf41bc57a826e75999fe370d6d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee0eecdf41bc57a826e75999fe370d6d columnFamilyName info 2023-08-07 13:56:25,244 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK] 2023-08-07 13:56:25,244 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK] 2023-08-07 13:56:25,246 DEBUG [StoreOpener-ee0eecdf41bc57a826e75999fe370d6d-1] regionserver.HStore(539): loaded hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d/info/0585593dc6864458ab6dab9850300c9f 2023-08-07 13:56:25,279 INFO [StoreOpener-ee0eecdf41bc57a826e75999fe370d6d-1] regionserver.HStore(310): Store=ee0eecdf41bc57a826e75999fe370d6d/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:25,281 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:56:25,283 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:56:25,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1 for peer 1 2023-08-07 13:56:25,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,34615,1691416579859/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1.1691416585222 2023-08-07 13:56:25,291 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:56:25,292 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ee0eecdf41bc57a826e75999fe370d6d; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67989558, jitterRate=0.013123363256454468}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 13:56:25,292 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ee0eecdf41bc57a826e75999fe370d6d: 2023-08-07 13:56:25,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1 2023-08-07 13:56:25,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-07 13:56:25,294 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d., pid=27, masterSystemTime=1691416585210 2023-08-07 13:56:25,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46767,DS-babc7b69-da91-489f-ab02-fda897fb6aca,DISK], DatanodeInfoWithStorage[127.0.0.1:33541,DS-8a538a5a-921c-432a-ad90-b37a7c232eae,DISK]] 2023-08-07 13:56:25,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1 2023-08-07 13:56:25,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => baa9fe21942fd12dea5caf4ddaa33efe, NAME => 't2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:56:25,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:56:25,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:56:25,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:56:25,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:56:25,308 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=ee0eecdf41bc57a826e75999fe370d6d, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:25,308 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:56:25,308 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:56:25,309 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691416585308"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416585308"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416585308"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416585308"}]},"ts":"1691416585308"} 2023-08-07 13:56:25,311 INFO [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:56:25,313 DEBUG [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/cf1 2023-08-07 13:56:25,313 DEBUG [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/cf1 2023-08-07 13:56:25,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:25,314 INFO [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region baa9fe21942fd12dea5caf4ddaa33efe columnFamilyName cf1 2023-08-07 13:56:25,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=24 2023-08-07 13:56:25,320 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=24, state=SUCCESS; OpenRegionProcedure ee0eecdf41bc57a826e75999fe370d6d, server=jenkins-hbase4.apache.org,34615,1691416579859 in 268 msec 2023-08-07 13:56:25,324 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:25,324 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-07 13:56:25,325 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=19 2023-08-07 13:56:25,325 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=ee0eecdf41bc57a826e75999fe370d6d, ASSIGN in 281 msec 2023-08-07 13:56:25,325 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,46383,1691416540182 after splitting done 2023-08-07 13:56:25,325 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,46383,1691416540182 from processing; numProcessing=1 2023-08-07 13:56:25,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,46383,1691416540182, splitWal=true, meta=false in 4.8780 sec 2023-08-07 13:56:25,343 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,34615,1691416579859/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0.1691416580708, lastWalPosition=422, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-07 13:56:25,343 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:25,366 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 17cd2797e57d428da452894ee4464bca 2023-08-07 13:56:25,367 DEBUG [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] regionserver.HStore(539): loaded hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/cf1/17cd2797e57d428da452894ee4464bca 2023-08-07 13:56:25,367 INFO [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] regionserver.HStore(310): Store=baa9fe21942fd12dea5caf4ddaa33efe/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:25,367 INFO [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:56:25,369 DEBUG [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/norep 2023-08-07 13:56:25,369 DEBUG [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/norep 2023-08-07 13:56:25,370 INFO [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region baa9fe21942fd12dea5caf4ddaa33efe columnFamilyName norep 2023-08-07 13:56:25,422 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f7d8ee36c40b4ad5b67c550fef4a0b08 2023-08-07 13:56:25,422 DEBUG [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] regionserver.HStore(539): loaded hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/norep/f7d8ee36c40b4ad5b67c550fef4a0b08 2023-08-07 13:56:25,431 INFO [StoreOpener-baa9fe21942fd12dea5caf4ddaa33efe-1] regionserver.HStore(310): Store=baa9fe21942fd12dea5caf4ddaa33efe/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:25,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:56:25,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:56:25,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 83 2023-08-07 13:56:25,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 13:56:25,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:56:25,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened baa9fe21942fd12dea5caf4ddaa33efe; next sequenceid=210; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74481705, jitterRate=0.10986389219760895}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 13:56:25,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for baa9fe21942fd12dea5caf4ddaa33efe: 2023-08-07 13:56:25,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe., pid=25, masterSystemTime=1691416585210 2023-08-07 13:56:25,458 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:25,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:56:25,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:56:25,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:56:25,459 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=baa9fe21942fd12dea5caf4ddaa33efe, regionState=OPEN, repBarrier=210, openSeqNum=210, regionLocation=jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:25,459 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416585459"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416585459"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416585459"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416585459"}]},"ts":"1691416585459"} 2023-08-07 13:56:25,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e1cfded3452bdf9174e1f916e4d8e3f3, NAME => 't1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:56:25,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:56:25,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:56:25,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:56:25,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:56:25,463 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=23 2023-08-07 13:56:25,463 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=23, state=SUCCESS; OpenRegionProcedure baa9fe21942fd12dea5caf4ddaa33efe, server=jenkins-hbase4.apache.org,34615,1691416579859 in 415 msec 2023-08-07 13:56:25,464 INFO [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:56:25,465 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=baa9fe21942fd12dea5caf4ddaa33efe, ASSIGN in 428 msec 2023-08-07 13:56:25,474 DEBUG [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/cf1 2023-08-07 13:56:25,474 DEBUG [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/cf1 2023-08-07 13:56:25,474 INFO [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e1cfded3452bdf9174e1f916e4d8e3f3 columnFamilyName cf1 2023-08-07 13:56:25,497 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7d95de56b4fb4170b2c83aba3569728b 2023-08-07 13:56:25,497 DEBUG [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] regionserver.HStore(539): loaded hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/cf1/7d95de56b4fb4170b2c83aba3569728b 2023-08-07 13:56:25,497 INFO [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] regionserver.HStore(310): Store=e1cfded3452bdf9174e1f916e4d8e3f3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:25,497 INFO [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:56:25,499 DEBUG [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/norep 2023-08-07 13:56:25,499 DEBUG [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/norep 2023-08-07 13:56:25,500 INFO [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e1cfded3452bdf9174e1f916e4d8e3f3 columnFamilyName norep 2023-08-07 13:56:25,524 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for af3a3d1432f64ef9a61da51970b3a8cd 2023-08-07 13:56:25,525 DEBUG [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] regionserver.HStore(539): loaded hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/norep/af3a3d1432f64ef9a61da51970b3a8cd 2023-08-07 13:56:25,525 INFO [StoreOpener-e1cfded3452bdf9174e1f916e4d8e3f3-1] regionserver.HStore(310): Store=e1cfded3452bdf9174e1f916e4d8e3f3/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:25,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:56:25,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:56:25,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 13:56:25,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:56:25,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e1cfded3452bdf9174e1f916e4d8e3f3; next sequenceid=110; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65156502, jitterRate=-0.029092460870742798}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 13:56:25,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e1cfded3452bdf9174e1f916e4d8e3f3: 2023-08-07 13:56:25,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3., pid=26, masterSystemTime=1691416585210 2023-08-07 13:56:25,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:56:25,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:56:25,543 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=e1cfded3452bdf9174e1f916e4d8e3f3, regionState=OPEN, repBarrier=110, openSeqNum=110, regionLocation=jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:25,543 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416585543"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416585543"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416585543"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416585543"}]},"ts":"1691416585543"} 2023-08-07 13:56:25,549 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=22 2023-08-07 13:56:25,549 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=22, state=SUCCESS; OpenRegionProcedure e1cfded3452bdf9174e1f916e4d8e3f3, server=jenkins-hbase4.apache.org,34615,1691416579859 in 499 msec 2023-08-07 13:56:25,551 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=18 2023-08-07 13:56:25,551 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,38873,1691416540136 after splitting done 2023-08-07 13:56:25,551 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=e1cfded3452bdf9174e1f916e4d8e3f3, ASSIGN in 514 msec 2023-08-07 13:56:25,551 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,38873,1691416540136 from processing; numProcessing=0 2023-08-07 13:56:25,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,38873,1691416540136, splitWal=true, meta=true in 5.1060 sec 2023-08-07 13:56:25,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-07 13:56:25,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-07 13:56:25,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-07 13:56:25,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,34615,1691416579859/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1.1691416585222, lastWalPosition=823, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-07 13:56:25,572 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:25,603 INFO [jenkins-hbase4:34615Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1]: currently replicating from: hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,34615,1691416579859/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1.1691416585222 at position: 823 walGroup [jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0]: currently replicating from: hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,34615,1691416579859/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0.1691416580708 at position: 422 2023-08-07 13:56:25,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-07 13:56:25,681 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:25,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-07 13:56:25,786 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:25,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-07 13:56:25,891 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:25,980 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-07 13:56:25,995 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:26,026 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/namespace 2023-08-07 13:56:26,040 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-08-07 13:56:26,043 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-08-07 13:56:26,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 6.135sec 2023-08-07 13:56:26,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-07 13:56:26,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-07 13:56:26,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-07 13:56:26,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44307,1691416579816-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-07 13:56:26,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44307,1691416579816-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-07 13:56:26,046 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-07 13:56:26,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-07 13:56:26,100 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:26,106 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(139): Connect 0x4216cf75 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:26,111 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@62867522, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:26,113 DEBUG [hconnection-0x56fe78ec-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:56:26,115 INFO [RS-EventLoopGroup-14-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37518, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:56:26,120 INFO [Listener at localhost/41247] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-08-07 13:56:26,120 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4216cf75 to 127.0.0.1:52488 2023-08-07 13:56:26,120 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:26,121 INFO [Listener at localhost/41247] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:44307 after: jenkins-hbase4.apache.org:44307 2023-08-07 13:56:26,122 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(139): Connect 0x2c8a80d5 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:26,125 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@996a4b0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:26,126 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x08fe00ad to 127.0.0.1:52488 2023-08-07 13:56:26,126 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:26,126 DEBUG [Listener at localhost/41247] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-07 13:56:26,126 DEBUG [Listener at localhost/41247] util.JVMClusterUtil(257): Found active master hash=723769340, stopped=false 2023-08-07 13:56:26,126 INFO [Listener at localhost/41247] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,43899,1691416553054 2023-08-07 13:56:26,129 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46513-0x101dfb6a1a1001d, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-07 13:56:26,129 INFO [Listener at localhost/41247] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 13:56:26,129 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-07 13:56:26,131 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:56:26,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46513-0x101dfb6a1a1001d, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:56:26,131 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3ca3d84d to 127.0.0.1:52488 2023-08-07 13:56:26,132 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:26,132 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:56:26,132 INFO [Listener at localhost/41247] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46513,1691416553082' ***** 2023-08-07 13:56:26,132 INFO [Listener at localhost/41247] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 13:56:26,132 INFO [RS:0;jenkins-hbase4:46513] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 13:56:26,132 INFO [RS:0;jenkins-hbase4:46513] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 13:56:26,132 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 13:56:26,132 INFO [RS:0;jenkins-hbase4:46513] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 13:56:26,133 INFO [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(3305): Received CLOSE for 77564e07be12a17717602214bc11ceb0 2023-08-07 13:56:26,133 INFO [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(3305): Received CLOSE for 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:56:26,133 INFO [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(3305): Received CLOSE for 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:56:26,133 INFO [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:56:26,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 77564e07be12a17717602214bc11ceb0, disabling compactions & flushes 2023-08-07 13:56:26,134 DEBUG [RS:0;jenkins-hbase4:46513] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3b13b2a5 to 127.0.0.1:52488 2023-08-07 13:56:26,134 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:56:26,134 DEBUG [RS:0;jenkins-hbase4:46513] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:26,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:56:26,134 INFO [RS:0;jenkins-hbase4:46513] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 13:56:26,134 INFO [RS:0;jenkins-hbase4:46513] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 13:56:26,134 INFO [RS:0;jenkins-hbase4:46513] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 13:56:26,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. after waiting 0 ms 2023-08-07 13:56:26,134 INFO [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-07 13:56:26,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:56:26,135 INFO [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-08-07 13:56:26,135 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 77564e07be12a17717602214bc11ceb0 2/2 column families, dataSize=1.36 KB heapSize=5.58 KB 2023-08-07 13:56:26,135 DEBUG [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, 77564e07be12a17717602214bc11ceb0=t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0., 3bc256268a4ab1eec2fd5f34b09fb147=hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147., 75110bd10fe307a72d13190ea044da7a=t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.} 2023-08-07 13:56:26,135 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 13:56:26,135 DEBUG [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(1504): Waiting on 1588230740, 3bc256268a4ab1eec2fd5f34b09fb147, 75110bd10fe307a72d13190ea044da7a, 77564e07be12a17717602214bc11ceb0 2023-08-07 13:56:26,135 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 13:56:26,136 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 13:56:26,136 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 13:56:26,136 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 13:56:26,136 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-08-07 13:56:26,151 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/.tmp/info/bf302993307c47d9bfad190ab0ffb485 2023-08-07 13:56:26,152 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.36 KB at sequenceid=13 (bloomFilter=true), to=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/.tmp/cf1/681f5294d01f4af78e7bb7068cc0d77e 2023-08-07 13:56:26,157 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/.tmp/info/bf302993307c47d9bfad190ab0ffb485 as hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info/bf302993307c47d9bfad190ab0ffb485 2023-08-07 13:56:26,157 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 681f5294d01f4af78e7bb7068cc0d77e 2023-08-07 13:56:26,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/.tmp/cf1/681f5294d01f4af78e7bb7068cc0d77e as hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1/681f5294d01f4af78e7bb7068cc0d77e 2023-08-07 13:56:26,162 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info/bf302993307c47d9bfad190ab0ffb485, entries=24, sequenceid=31, filesize=7.4 K 2023-08-07 13:56:26,163 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 681f5294d01f4af78e7bb7068cc0d77e 2023-08-07 13:56:26,163 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1/681f5294d01f4af78e7bb7068cc0d77e, entries=50, sequenceid=13, filesize=6.4 K 2023-08-07 13:56:26,164 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.36 KB/1390, heapSize ~5.31 KB/5440, currentSize=0 B/0 for 77564e07be12a17717602214bc11ceb0 in 30ms, sequenceid=13, compaction requested=false 2023-08-07 13:56:26,166 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 28ms, sequenceid=31, compaction requested=false 2023-08-07 13:56:26,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/recovered.edits/16.seqid, newMaxSeqId=16, maxSeqId=9 2023-08-07 13:56:26,184 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-08-07 13:56:26,185 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:56:26,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 77564e07be12a17717602214bc11ceb0: 2023-08-07 13:56:26,185 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 13:56:26,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:56:26,186 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 13:56:26,186 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 13:56:26,187 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3bc256268a4ab1eec2fd5f34b09fb147, disabling compactions & flushes 2023-08-07 13:56:26,187 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-07 13:56:26,187 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:56:26,187 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:56:26,187 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. after waiting 0 ms 2023-08-07 13:56:26,187 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:56:26,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-07 13:56:26,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-08-07 13:56:26,196 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:56:26,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3bc256268a4ab1eec2fd5f34b09fb147: 2023-08-07 13:56:26,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:56:26,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 75110bd10fe307a72d13190ea044da7a, disabling compactions & flushes 2023-08-07 13:56:26,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:56:26,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:56:26,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. after waiting 0 ms 2023-08-07 13:56:26,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:56:26,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 75110bd10fe307a72d13190ea044da7a 2/2 column families, dataSize=2.72 KB heapSize=10.66 KB 2023-08-07 13:56:26,204 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:26,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=16 (bloomFilter=true), to=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/.tmp/cf1/1dbe1760723d4f519052b5fc9df408e5 2023-08-07 13:56:26,214 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1dbe1760723d4f519052b5fc9df408e5 2023-08-07 13:56:26,215 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/.tmp/cf1/1dbe1760723d4f519052b5fc9df408e5 as hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1/1dbe1760723d4f519052b5fc9df408e5 2023-08-07 13:56:26,215 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:56:26,221 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1dbe1760723d4f519052b5fc9df408e5 2023-08-07 13:56:26,221 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1/1dbe1760723d4f519052b5fc9df408e5, entries=100, sequenceid=16, filesize=7.9 K 2023-08-07 13:56:26,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2790, heapSize ~10.39 KB/10640, currentSize=0 B/0 for 75110bd10fe307a72d13190ea044da7a in 25ms, sequenceid=16, compaction requested=false 2023-08-07 13:56:26,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/recovered.edits/19.seqid, newMaxSeqId=19, maxSeqId=12 2023-08-07 13:56:26,233 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:56:26,233 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 75110bd10fe307a72d13190ea044da7a: 2023-08-07 13:56:26,233 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:56:26,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-07 13:56:26,308 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:26,336 INFO [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46513,1691416553082; all regions closed. 2023-08-07 13:56:26,341 DEBUG [RS:0;jenkins-hbase4:46513] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs 2023-08-07 13:56:26,341 INFO [RS:0;jenkins-hbase4:46513] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46513%2C1691416553082.meta:.meta(num 1691416553633) 2023-08-07 13:56:26,345 DEBUG [RS:0;jenkins-hbase4:46513] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs 2023-08-07 13:56:26,345 INFO [RS:0;jenkins-hbase4:46513] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46513%2C1691416553082.jenkins-hbase4.apache.org%2C46513%2C1691416553082.regiongroup-1:(num 1691416558182) 2023-08-07 13:56:26,349 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46513,1691416553082/jenkins-hbase4.apache.org%2C46513%2C1691416553082.jenkins-hbase4.apache.org%2C46513%2C1691416553082.regiongroup-0.1691416553484 not finished, retry = 0 2023-08-07 13:56:26,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-07 13:56:26,414 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:26,414 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-07 13:56:26,414 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-07 13:56:26,453 DEBUG [RS:0;jenkins-hbase4:46513] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs 2023-08-07 13:56:26,453 INFO [RS:0;jenkins-hbase4:46513] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46513%2C1691416553082.jenkins-hbase4.apache.org%2C46513%2C1691416553082.regiongroup-0:(num 1691416553484) 2023-08-07 13:56:26,453 DEBUG [RS:0;jenkins-hbase4:46513] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:26,453 INFO [RS:0;jenkins-hbase4:46513] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:56:26,453 INFO [RS:0;jenkins-hbase4:46513] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-07 13:56:26,454 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 13:56:26,454 DEBUG [RS:0;jenkins-hbase4:46513] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3c5b0eb3 to 127.0.0.1:52488 2023-08-07 13:56:26,454 DEBUG [RS:0;jenkins-hbase4:46513] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:26,456 INFO [RS:0;jenkins-hbase4:46513] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46513 2023-08-07 13:56:26,460 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46513-0x101dfb6a1a1001d, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,46513,1691416553082 2023-08-07 13:56:26,460 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a10026, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:56:26,460 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46513-0x101dfb6a1a1001d, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:56:26,460 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-07 13:56:26,460 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:56:26,461 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46513,1691416553082] 2023-08-07 13:56:26,461 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46513,1691416553082; numProcessing=1 2023-08-07 13:56:26,463 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46513,1691416553082 already deleted, retry=false 2023-08-07 13:56:26,463 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46513,1691416553082 expired; onlineServers=0 2023-08-07 13:56:26,463 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43899,1691416553054' ***** 2023-08-07 13:56:26,463 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-07 13:56:26,464 DEBUG [M:0;jenkins-hbase4:43899] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f1c5640, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 13:56:26,464 INFO [M:0;jenkins-hbase4:43899] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43899,1691416553054 2023-08-07 13:56:26,464 INFO [M:0;jenkins-hbase4:43899] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43899,1691416553054; all regions closed. 2023-08-07 13:56:26,464 DEBUG [M:0;jenkins-hbase4:43899] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:26,464 DEBUG [M:0;jenkins-hbase4:43899] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-07 13:56:26,464 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-07 13:56:26,464 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416553295] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416553295,5,FailOnTimeoutGroup] 2023-08-07 13:56:26,464 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416553298] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416553298,5,FailOnTimeoutGroup] 2023-08-07 13:56:26,464 DEBUG [M:0;jenkins-hbase4:43899] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-07 13:56:26,466 INFO [M:0;jenkins-hbase4:43899] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-07 13:56:26,466 INFO [M:0;jenkins-hbase4:43899] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-07 13:56:26,466 INFO [M:0;jenkins-hbase4:43899] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-07 13:56:26,466 DEBUG [M:0;jenkins-hbase4:43899] master.HMaster(1512): Stopping service threads 2023-08-07 13:56:26,466 INFO [M:0;jenkins-hbase4:43899] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-07 13:56:26,468 ERROR [M:0;jenkins-hbase4:43899] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 13:56:26,468 INFO [M:0;jenkins-hbase4:43899] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 13:56:26,468 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-07 13:56:26,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-07 13:56:26,519 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:26,562 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46513-0x101dfb6a1a1001d, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:26,562 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:46513-0x101dfb6a1a1001d, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:26,562 INFO [RS:0;jenkins-hbase4:46513] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46513,1691416553082; zookeeper connection closed. 2023-08-07 13:56:26,564 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1cfd8fa5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1cfd8fa5 2023-08-07 13:56:26,564 INFO [Listener at localhost/41247] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-07 13:56:26,566 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-08-07 13:56:26,566 INFO [M:0;jenkins-hbase4:43899] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 13:56:26,566 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:56:26,567 INFO [M:0;jenkins-hbase4:43899] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-07 13:56:26,568 DEBUG [M:0;jenkins-hbase4:43899] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 13:56:26,568 INFO [M:0;jenkins-hbase4:43899] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:56:26,568 DEBUG [M:0;jenkins-hbase4:43899] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:56:26,569 DEBUG [M:0;jenkins-hbase4:43899] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 13:56:26,569 DEBUG [M:0;jenkins-hbase4:43899] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:56:26,569 INFO [M:0;jenkins-hbase4:43899] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=36.70 KB heapSize=47.15 KB 2023-08-07 13:56:26,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-07 13:56:26,570 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/master already deleted, retry=false 2023-08-07 13:56:26,571 DEBUG [RegionServerTracker-0] master.ActiveMasterManager(335): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Failed delete of our master address node; KeeperErrorCode = NoNode for /2/master 2023-08-07 13:56:26,603 INFO [M:0;jenkins-hbase4:43899] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=36.70 KB at sequenceid=239 (bloomFilter=true), to=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e137dd3340314759a6d1597039c1ed20 2023-08-07 13:56:26,612 INFO [M:0;jenkins-hbase4:43899] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e137dd3340314759a6d1597039c1ed20 2023-08-07 13:56:26,614 DEBUG [M:0;jenkins-hbase4:43899] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e137dd3340314759a6d1597039c1ed20 as hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e137dd3340314759a6d1597039c1ed20 2023-08-07 13:56:26,616 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-07 13:56:26,621 INFO [M:0;jenkins-hbase4:43899] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e137dd3340314759a6d1597039c1ed20 2023-08-07 13:56:26,622 INFO [M:0;jenkins-hbase4:43899] regionserver.HStore(1080): Added hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e137dd3340314759a6d1597039c1ed20, entries=20, sequenceid=239, filesize=5.6 K 2023-08-07 13:56:26,623 INFO [M:0;jenkins-hbase4:43899] regionserver.HRegion(2948): Finished flush of dataSize ~36.70 KB/37578, heapSize ~47.13 KB/48264, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 54ms, sequenceid=239, compaction requested=false 2023-08-07 13:56:26,652 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:26,657 INFO [M:0;jenkins-hbase4:43899] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:56:26,657 DEBUG [M:0;jenkins-hbase4:43899] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 13:56:26,679 INFO [M:0;jenkins-hbase4:43899] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-07 13:56:26,679 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 13:56:26,680 INFO [M:0;jenkins-hbase4:43899] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43899 2023-08-07 13:56:26,692 DEBUG [M:0;jenkins-hbase4:43899] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,43899,1691416553054 already deleted, retry=false 2023-08-07 13:56:26,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-08-07 13:56:26,766 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:26,794 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:26,794 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:43899-0x101dfb6a1a1001c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:26,794 INFO [M:0;jenkins-hbase4:43899] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43899,1691416553054; zookeeper connection closed. 2023-08-07 13:56:26,799 DEBUG [hconnection-0x6eecfed7-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:56:26,801 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37528, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:56:26,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 2063 2023-08-07 13:56:26,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 12 WAL entries eligible for replication 2023-08-07 13:56:26,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 2063 2023-08-07 13:56:26,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/112=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/113=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/114=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/115=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/116=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/117=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/118=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/119=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/120=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/121=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/122=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/123=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,34615,1691416579859/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1.1691416585222, lastWalPosition=2063, nbRowKeys=12, nbHFiles=0, heapSize=2496, lastSeqIds={}, endOfFile=false,usedBufferSize=2496] 2023-08-07 13:56:26,846 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-07 13:56:26,875 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:26,946 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-07 13:56:26,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 7359 2023-08-07 13:56:26,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 50 WAL entries eligible for replication 2023-08-07 13:56:26,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 7359 2023-08-07 13:56:26,983 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:27,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14553 2023-08-07 13:56:27,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 67 WAL entries eligible for replication 2023-08-07 13:56:27,092 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:27,147 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-07 13:56:27,198 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:27,310 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-08-07 13:56:27,342 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2c8a80d5 to 127.0.0.1:52488 2023-08-07 13:56:27,342 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:27,342 DEBUG [Listener at localhost/41247] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-07 13:56:27,342 DEBUG [Listener at localhost/41247] util.JVMClusterUtil(257): Found active master hash=1648461822, stopped=false 2023-08-07 13:56:27,342 INFO [Listener at localhost/41247] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,44307,1691416579816 2023-08-07 13:56:27,345 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34615-0x101dfb6a1a1002b, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-07 13:56:27,345 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-08-07 13:56:27,345 INFO [Listener at localhost/41247] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 13:56:27,345 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:56:27,348 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5da4278d to 127.0.0.1:52488 2023-08-07 13:56:27,349 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:27,349 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-07 13:56:27,348 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34615-0x101dfb6a1a1002b, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-08-07 13:56:27,350 INFO [Listener at localhost/41247] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34615,1691416579859' ***** 2023-08-07 13:56:27,351 INFO [Listener at localhost/41247] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 13:56:27,351 INFO [RS:0;jenkins-hbase4:34615] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 13:56:27,351 INFO [RS:0;jenkins-hbase4:34615] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 13:56:27,351 INFO [RS:0;jenkins-hbase4:34615] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 13:56:27,351 INFO [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(3305): Received CLOSE for ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:56:27,352 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 13:56:27,369 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:56:27,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ee0eecdf41bc57a826e75999fe370d6d, disabling compactions & flushes 2023-08-07 13:56:27,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:56:27,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:56:27,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. after waiting 0 ms 2023-08-07 13:56:27,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:56:27,374 INFO [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(3305): Received CLOSE for baa9fe21942fd12dea5caf4ddaa33efe 2023-08-07 13:56:27,375 INFO [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(3305): Received CLOSE for e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:56:27,375 INFO [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:27,375 DEBUG [RS:0;jenkins-hbase4:34615] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6b81ca89 to 127.0.0.1:52488 2023-08-07 13:56:27,375 DEBUG [RS:0;jenkins-hbase4:34615] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:27,375 INFO [RS:0;jenkins-hbase4:34615] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 13:56:27,375 INFO [RS:0;jenkins-hbase4:34615] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 13:56:27,375 INFO [RS:0;jenkins-hbase4:34615] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 13:56:27,375 INFO [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-07 13:56:27,378 INFO [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-08-07 13:56:27,379 DEBUG [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, ee0eecdf41bc57a826e75999fe370d6d=hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d., baa9fe21942fd12dea5caf4ddaa33efe=t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe., e1cfded3452bdf9174e1f916e4d8e3f3=t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3.} 2023-08-07 13:56:27,379 DEBUG [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(1504): Waiting on 1588230740, baa9fe21942fd12dea5caf4ddaa33efe, e1cfded3452bdf9174e1f916e4d8e3f3, ee0eecdf41bc57a826e75999fe370d6d 2023-08-07 13:56:27,383 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 13:56:27,384 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 13:56:27,384 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 13:56:27,384 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 13:56:27,384 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 13:56:27,384 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.94 KB heapSize=5.62 KB 2023-08-07 13:56:27,402 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/namespace/ee0eecdf41bc57a826e75999fe370d6d/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-08-07 13:56:27,403 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:56:27,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ee0eecdf41bc57a826e75999fe370d6d: 2023-08-07 13:56:27,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691416542218.ee0eecdf41bc57a826e75999fe370d6d. 2023-08-07 13:56:27,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing baa9fe21942fd12dea5caf4ddaa33efe, disabling compactions & flushes 2023-08-07 13:56:27,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:56:27,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:56:27,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. after waiting 0 ms 2023-08-07 13:56:27,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:56:27,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing baa9fe21942fd12dea5caf4ddaa33efe 2/2 column families, dataSize=7.05 KB heapSize=22.49 KB 2023-08-07 13:56:27,431 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/.tmp/info/3293ae2fa529477aa85be1ca926bcdd8 2023-08-07 13:56:27,448 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-08-07 13:56:27,448 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-08-07 13:56:27,448 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-08-07 13:56:27,448 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,34615,1691416579859/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0.1691416580708, lastWalPosition=728, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-07 13:56:27,463 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/.tmp/cf1/f14b82859f4c4ac6924ddcf2e81bd0a7 2023-08-07 13:56:27,480 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/.tmp/rep_barrier/ae7503cf133247df9b2050f6afc0d14f 2023-08-07 13:56:27,481 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-08-07 13:56:27,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/.tmp/norep/724bc463a82c430887c56a0e8f7c502f 2023-08-07 13:56:27,490 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/.tmp/info/3293ae2fa529477aa85be1ca926bcdd8 as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/info/3293ae2fa529477aa85be1ca926bcdd8 2023-08-07 13:56:27,502 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/info/3293ae2fa529477aa85be1ca926bcdd8, entries=24, sequenceid=31, filesize=7.4 K 2023-08-07 13:56:27,503 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/.tmp/rep_barrier/ae7503cf133247df9b2050f6afc0d14f as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/rep_barrier/ae7503cf133247df9b2050f6afc0d14f 2023-08-07 13:56:27,507 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/.tmp/cf1/f14b82859f4c4ac6924ddcf2e81bd0a7 as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/cf1/f14b82859f4c4ac6924ddcf2e81bd0a7 2023-08-07 13:56:27,509 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/rep_barrier/ae7503cf133247df9b2050f6afc0d14f, entries=2, sequenceid=31, filesize=4.9 K 2023-08-07 13:56:27,511 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.94 KB/3012, heapSize ~5.34 KB/5464, currentSize=0 B/0 for 1588230740 in 126ms, sequenceid=31, compaction requested=false 2023-08-07 13:56:27,517 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/cf1/f14b82859f4c4ac6924ddcf2e81bd0a7, entries=200, sequenceid=413, filesize=12.5 K 2023-08-07 13:56:27,520 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/.tmp/norep/724bc463a82c430887c56a0e8f7c502f as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/norep/724bc463a82c430887c56a0e8f7c502f 2023-08-07 13:56:27,526 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/norep/724bc463a82c430887c56a0e8f7c502f, entries=1, sequenceid=413, filesize=4.8 K 2023-08-07 13:56:27,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.05 KB/7221, heapSize ~22.46 KB/23000, currentSize=0 B/0 for baa9fe21942fd12dea5caf4ddaa33efe in 122ms, sequenceid=413, compaction requested=false 2023-08-07 13:56:27,527 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-08-07 13:56:27,532 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 13:56:27,532 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 13:56:27,532 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 13:56:27,532 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-07 13:56:27,535 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/recovered.edits/416.seqid, newMaxSeqId=416, maxSeqId=209 2023-08-07 13:56:27,536 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:56:27,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for baa9fe21942fd12dea5caf4ddaa33efe: 2023-08-07 13:56:27,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1691416546829.baa9fe21942fd12dea5caf4ddaa33efe. 2023-08-07 13:56:27,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e1cfded3452bdf9174e1f916e4d8e3f3, disabling compactions & flushes 2023-08-07 13:56:27,537 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:56:27,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:56:27,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. after waiting 0 ms 2023-08-07 13:56:27,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:56:27,537 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing e1cfded3452bdf9174e1f916e4d8e3f3 2/2 column families, dataSize=3.44 KB heapSize=11.55 KB 2023-08-07 13:56:27,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/.tmp/cf1/5ba5260e8f0e4da4b1bae84b577e454f 2023-08-07 13:56:27,552 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-08-07 13:56:27,563 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-07 13:56:27,563 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-07 13:56:27,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/.tmp/norep/8e8c3516336446099f4ff9282e4c68ae 2023-08-07 13:56:27,570 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/.tmp/cf1/5ba5260e8f0e4da4b1bae84b577e454f as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/cf1/5ba5260e8f0e4da4b1bae84b577e454f 2023-08-07 13:56:27,575 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/cf1/5ba5260e8f0e4da4b1bae84b577e454f, entries=100, sequenceid=213, filesize=8.4 K 2023-08-07 13:56:27,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/.tmp/norep/8e8c3516336446099f4ff9282e4c68ae as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/norep/8e8c3516336446099f4ff9282e4c68ae 2023-08-07 13:56:27,579 DEBUG [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(1504): Waiting on e1cfded3452bdf9174e1f916e4d8e3f3 2023-08-07 13:56:27,581 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/norep/8e8c3516336446099f4ff9282e4c68ae, entries=1, sequenceid=213, filesize=4.8 K 2023-08-07 13:56:27,582 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.44 KB/3521, heapSize ~11.52 KB/11800, currentSize=0 B/0 for e1cfded3452bdf9174e1f916e4d8e3f3 in 45ms, sequenceid=213, compaction requested=false 2023-08-07 13:56:27,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/data/default/t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/recovered.edits/216.seqid, newMaxSeqId=216, maxSeqId=109 2023-08-07 13:56:27,591 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:56:27,591 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e1cfded3452bdf9174e1f916e4d8e3f3: 2023-08-07 13:56:27,591 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1691416546190.e1cfded3452bdf9174e1f916e4d8e3f3. 2023-08-07 13:56:27,658 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-08-07 13:56:27,774 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-08-07 13:56:27,779 INFO [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34615,1691416579859; all regions closed. 2023-08-07 13:56:27,787 DEBUG [RS:0;jenkins-hbase4:34615] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs 2023-08-07 13:56:27,787 INFO [RS:0;jenkins-hbase4:34615] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34615%2C1691416579859.meta:.meta(num 1691416580816) 2023-08-07 13:56:27,794 DEBUG [RS:0;jenkins-hbase4:34615] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs 2023-08-07 13:56:27,794 INFO [RS:0;jenkins-hbase4:34615] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0:(num 1691416580708) 2023-08-07 13:56:27,800 DEBUG [RS:0;jenkins-hbase4:34615] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs 2023-08-07 13:56:27,800 INFO [RS:0;jenkins-hbase4:34615] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1:(num 1691416585222) 2023-08-07 13:56:27,801 DEBUG [RS:0;jenkins-hbase4:34615] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:27,801 INFO [RS:0;jenkins-hbase4:34615] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:56:27,801 INFO [RS:0;jenkins-hbase4:34615] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-07 13:56:27,801 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 13:56:27,801 INFO [RS:0;jenkins-hbase4:34615] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-07 13:56:27,876 INFO [RS:0;jenkins-hbase4:34615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/WALs/jenkins-hbase4.apache.org,34615,1691416579859/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0.1691416580708 was moved to hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0.1691416580708 2023-08-07 13:56:27,882 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(515): No replication sinks found, returning without replicating. The source should retry with the same set of edits. Not logging this again for the next 10 seconds. 2023-08-07 13:56:27,905 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a1002f, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:27,905 DEBUG [RS:0;jenkins-hbase4:34615] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1e287bc8 to 127.0.0.1:52488 2023-08-07 13:56:27,905 DEBUG [RS:0;jenkins-hbase4:34615.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a1002f, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:27,905 DEBUG [RS:0;jenkins-hbase4:34615] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:28,006 INFO [RS:0;jenkins-hbase4:34615] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1 terminated 2023-08-07 13:56:28,008 INFO [RS:0;jenkins-hbase4:34615] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS:0;jenkins-hbase4:34615.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1 terminated 2023-08-07 13:56:28,010 INFO [RS:0;jenkins-hbase4:34615] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34615 2023-08-07 13:56:28,015 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34615-0x101dfb6a1a1002b, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:28,015 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-07 13:56:28,015 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34615-0x101dfb6a1a1002b, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-08-07 13:56:28,017 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34615,1691416579859] 2023-08-07 13:56:28,017 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34615,1691416579859; numProcessing=1 2023-08-07 13:56:28,018 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,34615,1691416579859 already deleted, retry=false 2023-08-07 13:56:28,018 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34615,1691416579859 expired; onlineServers=0 2023-08-07 13:56:28,018 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,44307,1691416579816' ***** 2023-08-07 13:56:28,018 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-07 13:56:28,019 DEBUG [M:0;jenkins-hbase4:44307] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@667e1f6e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 13:56:28,019 INFO [M:0;jenkins-hbase4:44307] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,44307,1691416579816 2023-08-07 13:56:28,019 INFO [M:0;jenkins-hbase4:44307] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,44307,1691416579816; all regions closed. 2023-08-07 13:56:28,019 DEBUG [M:0;jenkins-hbase4:44307] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:28,019 DEBUG [M:0;jenkins-hbase4:44307] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-07 13:56:28,019 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-07 13:56:28,019 DEBUG [M:0;jenkins-hbase4:44307] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-07 13:56:28,019 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416580486] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416580486,5,FailOnTimeoutGroup] 2023-08-07 13:56:28,020 INFO [M:0;jenkins-hbase4:44307] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-07 13:56:28,020 INFO [M:0;jenkins-hbase4:44307] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-07 13:56:28,019 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416580486] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416580486,5,FailOnTimeoutGroup] 2023-08-07 13:56:28,021 INFO [M:0;jenkins-hbase4:44307] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-07 13:56:28,021 DEBUG [M:0;jenkins-hbase4:44307] master.HMaster(1512): Stopping service threads 2023-08-07 13:56:28,021 INFO [M:0;jenkins-hbase4:44307] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-07 13:56:28,022 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-08-07 13:56:28,022 ERROR [M:0;jenkins-hbase4:44307] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 13:56:28,022 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-08-07 13:56:28,022 INFO [M:0;jenkins-hbase4:44307] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 13:56:28,022 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-07 13:56:28,022 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-08-07 13:56:28,023 DEBUG [M:0;jenkins-hbase4:44307] zookeeper.ZKUtil(398): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-08-07 13:56:28,023 WARN [M:0;jenkins-hbase4:44307] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-07 13:56:28,023 INFO [M:0;jenkins-hbase4:44307] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 13:56:28,023 INFO [M:0;jenkins-hbase4:44307] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-07 13:56:28,025 DEBUG [M:0;jenkins-hbase4:44307] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 13:56:28,025 INFO [M:0;jenkins-hbase4:44307] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:56:28,025 DEBUG [M:0;jenkins-hbase4:44307] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:56:28,025 DEBUG [M:0;jenkins-hbase4:44307] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 13:56:28,025 DEBUG [M:0;jenkins-hbase4:44307] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:56:28,025 INFO [M:0;jenkins-hbase4:44307] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=29.71 KB heapSize=36.95 KB 2023-08-07 13:56:28,036 INFO [M:0;jenkins-hbase4:44307] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=29.71 KB at sequenceid=218 (bloomFilter=true), to=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/c91a05c5a3bc4b2995976d8906fec07a 2023-08-07 13:56:28,042 DEBUG [M:0;jenkins-hbase4:44307] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/c91a05c5a3bc4b2995976d8906fec07a as hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/c91a05c5a3bc4b2995976d8906fec07a 2023-08-07 13:56:28,047 INFO [M:0;jenkins-hbase4:44307] regionserver.HStore(1080): Added hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/c91a05c5a3bc4b2995976d8906fec07a, entries=10, sequenceid=218, filesize=5.8 K 2023-08-07 13:56:28,048 INFO [M:0;jenkins-hbase4:44307] regionserver.HRegion(2948): Finished flush of dataSize ~29.71 KB/30423, heapSize ~36.94 KB/37824, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 23ms, sequenceid=218, compaction requested=false 2023-08-07 13:56:28,050 INFO [M:0;jenkins-hbase4:44307] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:56:28,050 DEBUG [M:0;jenkins-hbase4:44307] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 13:56:28,057 INFO [M:0;jenkins-hbase4:44307] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-07 13:56:28,057 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 13:56:28,058 INFO [M:0;jenkins-hbase4:44307] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44307 2023-08-07 13:56:28,060 DEBUG [M:0;jenkins-hbase4:44307] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,44307,1691416579816 already deleted, retry=false 2023-08-07 13:56:28,117 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34615-0x101dfb6a1a1002b, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:28,117 INFO [RS:0;jenkins-hbase4:34615] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34615,1691416579859; zookeeper connection closed. 2023-08-07 13:56:28,117 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:34615-0x101dfb6a1a1002b, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:28,119 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@58bb7af4] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@58bb7af4 2023-08-07 13:56:28,119 INFO [Listener at localhost/41247] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-07 13:56:28,217 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:28,217 INFO [M:0;jenkins-hbase4:44307] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,44307,1691416579816; zookeeper connection closed. 2023-08-07 13:56:28,217 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:44307-0x101dfb6a1a1002a, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:28,221 INFO [Listener at localhost/41247] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-08-07 13:56:28,221 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:56:28,222 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 13:56:28,222 INFO [Listener at localhost/41247] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 13:56:28,222 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:56:28,222 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 13:56:28,222 INFO [Listener at localhost/41247] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 13:56:28,223 INFO [Listener at localhost/41247] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35101 2023-08-07 13:56:28,223 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:56:28,225 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:56:28,226 INFO [Listener at localhost/41247] zookeeper.RecoverableZooKeeper(93): Process identifier=master:35101 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:56:28,229 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:351010x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:56:28,230 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:35101-0x101dfb6a1a10032 connected 2023-08-07 13:56:28,232 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-07 13:56:28,233 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:56:28,233 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-07 13:56:28,233 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35101 2023-08-07 13:56:28,234 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35101 2023-08-07 13:56:28,234 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35101 2023-08-07 13:56:28,237 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35101 2023-08-07 13:56:28,237 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35101 2023-08-07 13:56:28,237 INFO [Listener at localhost/41247] master.HMaster(444): hbase.rootdir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2, hbase.cluster.distributed=false 2023-08-07 13:56:28,238 DEBUG [pool-666-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-08-07 13:56:28,250 INFO [Listener at localhost/41247] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-08-07 13:56:28,250 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:56:28,250 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-07 13:56:28,251 INFO [Listener at localhost/41247] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-07 13:56:28,251 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-07 13:56:28,251 INFO [Listener at localhost/41247] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-07 13:56:28,251 INFO [Listener at localhost/41247] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-07 13:56:28,252 INFO [Listener at localhost/41247] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44621 2023-08-07 13:56:28,252 INFO [Listener at localhost/41247] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-07 13:56:28,256 DEBUG [Listener at localhost/41247] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-07 13:56:28,256 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:56:28,257 INFO [Listener at localhost/41247] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:56:28,258 INFO [Listener at localhost/41247] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44621 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:56:28,260 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:446210x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:56:28,263 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:44621-0x101dfb6a1a10033 connected 2023-08-07 13:56:28,263 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:44621-0x101dfb6a1a10033, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-07 13:56:28,263 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:44621-0x101dfb6a1a10033, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:56:28,264 DEBUG [Listener at localhost/41247] zookeeper.ZKUtil(164): regionserver:44621-0x101dfb6a1a10033, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-08-07 13:56:28,266 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44621 2023-08-07 13:56:28,266 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44621 2023-08-07 13:56:28,271 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44621 2023-08-07 13:56:28,271 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44621 2023-08-07 13:56:28,271 DEBUG [Listener at localhost/41247] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44621 2023-08-07 13:56:28,273 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,35101,1691416588221 2023-08-07 13:56:28,276 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-07 13:56:28,276 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,35101,1691416588221 2023-08-07 13:56:28,278 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-07 13:56:28,278 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:44621-0x101dfb6a1a10033, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-08-07 13:56:28,278 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:56:28,280 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-07 13:56:28,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/master 2023-08-07 13:56:28,280 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,35101,1691416588221 from backup master directory 2023-08-07 13:56:28,281 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,35101,1691416588221 2023-08-07 13:56:28,281 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-08-07 13:56:28,281 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 13:56:28,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,35101,1691416588221 2023-08-07 13:56:28,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-07 13:56:28,329 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x34afbec1 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:28,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2e26a0b0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:28,335 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-07 13:56:28,335 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-07 13:56:28,336 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:56:28,337 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:56:28,342 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,43899,1691416553054 to hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,43899,1691416553054-dead as it is dead 2023-08-07 13:56:28,342 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,43899,1691416553054-dead/jenkins-hbase4.apache.org%2C43899%2C1691416553054.jenkins-hbase4.apache.org%2C43899%2C1691416553054.regiongroup-0.1691416553174 2023-08-07 13:56:28,343 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,43899,1691416553054-dead/jenkins-hbase4.apache.org%2C43899%2C1691416553054.jenkins-hbase4.apache.org%2C43899%2C1691416553054.regiongroup-0.1691416553174 after 1ms 2023-08-07 13:56:28,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,43899,1691416553054-dead/jenkins-hbase4.apache.org%2C43899%2C1691416553054.jenkins-hbase4.apache.org%2C43899%2C1691416553054.regiongroup-0.1691416553174 to hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C43899%2C1691416553054.jenkins-hbase4.apache.org%2C43899%2C1691416553054.regiongroup-0.1691416553174 2023-08-07 13:56:28,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,43899,1691416553054-dead 2023-08-07 13:56:28,344 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,35101,1691416588221 2023-08-07 13:56:28,345 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:56:28,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35101%2C1691416588221.jenkins-hbase4.apache.org%2C35101%2C1691416588221.regiongroup-0, suffix=, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,35101,1691416588221, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/oldWALs, maxLogs=10 2023-08-07 13:56:28,365 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK] 2023-08-07 13:56:28,365 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK] 2023-08-07 13:56:28,366 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:56:28,371 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/WALs/jenkins-hbase4.apache.org,35101,1691416588221/jenkins-hbase4.apache.org%2C35101%2C1691416588221.jenkins-hbase4.apache.org%2C35101%2C1691416588221.regiongroup-0.1691416588347 2023-08-07 13:56:28,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK], DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK], DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK]] 2023-08-07 13:56:28,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:56:28,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:56:28,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:56:28,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:56:28,373 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:56:28,374 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-07 13:56:28,375 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-07 13:56:28,381 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5eec9c061c4641719002e9ed31c10ef2 2023-08-07 13:56:28,384 INFO [StoreFileOpener-proc-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e137dd3340314759a6d1597039c1ed20 2023-08-07 13:56:28,384 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e137dd3340314759a6d1597039c1ed20 2023-08-07 13:56:28,384 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:28,385 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-08-07 13:56:28,385 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C43899%2C1691416553054.jenkins-hbase4.apache.org%2C43899%2C1691416553054.regiongroup-0.1691416553174 2023-08-07 13:56:28,389 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 140, firstSequenceIdInLog=120, maxSequenceIdInLog=241, path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C43899%2C1691416553054.jenkins-hbase4.apache.org%2C43899%2C1691416553054.regiongroup-0.1691416553174 2023-08-07 13:56:28,390 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C43899%2C1691416553054.jenkins-hbase4.apache.org%2C43899%2C1691416553054.regiongroup-0.1691416553174 2023-08-07 13:56:28,393 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-07 13:56:28,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/241.seqid, newMaxSeqId=241, maxSeqId=118 2023-08-07 13:56:28,396 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=242; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65039892, jitterRate=-0.030830085277557373}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 13:56:28,396 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 13:56:28,396 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-07 13:56:28,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-07 13:56:28,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-07 13:56:28,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-07 13:56:28,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-07 13:56:28,405 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-07 13:56:28,405 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-07 13:56:28,406 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-07 13:56:28,406 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-07 13:56:28,407 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-08-07 13:56:28,407 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-08-07 13:56:28,407 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-07 13:56:28,407 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-07 13:56:28,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-07 13:56:28,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46513,1691416553082, table=hbase:meta, region=1588230740 2023-08-07 13:56:28,410 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 1 possibly 'live' servers, and 0 'splitting'. 2023-08-07 13:56:28,411 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46513,1691416553082 already deleted, retry=false 2023-08-07 13:56:28,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,46513,1691416553082 on jenkins-hbase4.apache.org,35101,1691416588221 2023-08-07 13:56:28,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,46513,1691416553082, splitWal=true, meta=true 2023-08-07 13:56:28,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=27 for jenkins-hbase4.apache.org,46513,1691416553082 (carryingMeta=true) jenkins-hbase4.apache.org,46513,1691416553082/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@77afac54[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-08-07 13:56:28,422 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-07 13:56:28,422 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-07 13:56:28,423 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-08-07 13:56:28,423 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-07 13:56:28,423 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-08-07 13:56:28,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-08-07 13:56:28,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-08-07 13:56:28,425 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-08-07 13:56:28,426 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-07 13:56:28,426 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:44621-0x101dfb6a1a10033, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-08-07 13:56:28,426 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:56:28,427 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,35101,1691416588221, sessionid=0x101dfb6a1a10032, setting cluster-up flag (Was=false) 2023-08-07 13:56:28,431 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-08-07 13:56:28,432 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,35101,1691416588221 2023-08-07 13:56:28,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-08-07 13:56:28,438 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,35101,1691416588221 2023-08-07 13:56:28,439 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/.hbase-snapshot/.tmp 2023-08-07 13:56:28,442 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:56:28,442 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:56:28,443 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:56:28,443 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-07 13:56:28,443 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-07 13:56:28,443 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:28,443 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 13:56:28,443 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:28,448 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1691416618448 2023-08-07 13:56:28,448 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-07 13:56:28,450 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-07 13:56:28,451 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-07 13:56:28,451 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-07 13:56:28,451 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-07 13:56:28,451 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-07 13:56:28,453 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46513,1691416553082; numProcessing=1 2023-08-07 13:56:28,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:28,453 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=27, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46513,1691416553082, splitWal=true, meta=true 2023-08-07 13:56:28,454 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-07 13:56:28,454 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-07 13:56:28,454 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-07 13:56:28,454 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46513,1691416553082, splitWal=true, meta=true, isMeta: true 2023-08-07 13:56:28,454 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-07 13:56:28,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-07 13:56:28,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416588455,5,FailOnTimeoutGroup] 2023-08-07 13:56:28,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416588455,5,FailOnTimeoutGroup] 2023-08-07 13:56:28,456 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:28,456 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-07 13:56:28,456 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:28,456 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:28,456 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1691416588456, completionTime=-1 2023-08-07 13:56:28,456 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-08-07 13:56:28,456 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-07 13:56:28,456 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46513,1691416553082-splitting 2023-08-07 13:56:28,457 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46513,1691416553082-splitting dir is empty, no logs to split. 2023-08-07 13:56:28,457 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46513,1691416553082 WAL count=0, meta=true 2023-08-07 13:56:28,459 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46513,1691416553082-splitting dir is empty, no logs to split. 2023-08-07 13:56:28,459 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46513,1691416553082 WAL count=0, meta=true 2023-08-07 13:56:28,459 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46513,1691416553082 WAL splitting is done? wals=0, meta=true 2023-08-07 13:56:28,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-07 13:56:28,461 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-07 13:56:28,463 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-07 13:56:28,472 INFO [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(951): ClusterId : bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:56:28,472 DEBUG [RS:0;jenkins-hbase4:44621] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-07 13:56:28,474 DEBUG [RS:0;jenkins-hbase4:44621] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-07 13:56:28,474 DEBUG [RS:0;jenkins-hbase4:44621] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-07 13:56:28,476 DEBUG [RS:0;jenkins-hbase4:44621] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-07 13:56:28,478 DEBUG [RS:0;jenkins-hbase4:44621] zookeeper.ReadOnlyZKClient(139): Connect 0x283715be to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:28,481 DEBUG [RS:0;jenkins-hbase4:44621] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@26f0fec8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:28,481 DEBUG [RS:0;jenkins-hbase4:44621] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@22cf7ece, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 13:56:28,490 DEBUG [RS:0;jenkins-hbase4:44621] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:44621 2023-08-07 13:56:28,490 INFO [RS:0;jenkins-hbase4:44621] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-07 13:56:28,490 INFO [RS:0;jenkins-hbase4:44621] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-07 13:56:28,490 DEBUG [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(1022): About to register with Master. 2023-08-07 13:56:28,490 INFO [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,35101,1691416588221 with isa=jenkins-hbase4.apache.org/172.31.14.131:44621, startcode=1691416588250 2023-08-07 13:56:28,491 DEBUG [RS:0;jenkins-hbase4:44621] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-07 13:56:28,492 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46105, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=RegionServerStatusService 2023-08-07 13:56:28,494 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=35101] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:28,494 DEBUG [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2 2023-08-07 13:56:28,494 DEBUG [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45993 2023-08-07 13:56:28,494 DEBUG [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-07 13:56:28,504 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a10026, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:56:28,504 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:56:28,504 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-07 13:56:28,505 DEBUG [RS:0;jenkins-hbase4:44621] zookeeper.ZKUtil(162): regionserver:44621-0x101dfb6a1a10033, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:28,505 WARN [RS:0;jenkins-hbase4:44621] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-07 13:56:28,506 INFO [RS:0;jenkins-hbase4:44621] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:56:28,506 INFO [RS:0;jenkins-hbase4:44621] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:56:28,506 DEBUG [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:28,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-08-07 13:56:28,506 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,44621,1691416588250] 2023-08-07 13:56:28,513 DEBUG [RS:0;jenkins-hbase4:44621] zookeeper.ZKUtil(162): regionserver:44621-0x101dfb6a1a10033, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:28,514 DEBUG [RS:0;jenkins-hbase4:44621] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-07 13:56:28,514 INFO [RS:0;jenkins-hbase4:44621] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-07 13:56:28,515 INFO [RS:0;jenkins-hbase4:44621] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-07 13:56:28,517 INFO [RS:0;jenkins-hbase4:44621] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-07 13:56:28,517 INFO [RS:0;jenkins-hbase4:44621] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:28,518 INFO [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-07 13:56:28,521 INFO [RS:0;jenkins-hbase4:44621] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:28,521 DEBUG [RS:0;jenkins-hbase4:44621] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:28,521 DEBUG [RS:0;jenkins-hbase4:44621] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:28,521 DEBUG [RS:0;jenkins-hbase4:44621] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:28,521 DEBUG [RS:0;jenkins-hbase4:44621] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:28,521 DEBUG [RS:0;jenkins-hbase4:44621] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:28,522 DEBUG [RS:0;jenkins-hbase4:44621] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-07 13:56:28,522 DEBUG [RS:0;jenkins-hbase4:44621] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:28,522 DEBUG [RS:0;jenkins-hbase4:44621] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:28,522 DEBUG [RS:0;jenkins-hbase4:44621] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:28,522 DEBUG [RS:0;jenkins-hbase4:44621] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-07 13:56:28,525 INFO [RS:0;jenkins-hbase4:44621] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:28,525 INFO [RS:0;jenkins-hbase4:44621] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:28,526 INFO [RS:0;jenkins-hbase4:44621] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:28,538 INFO [RS:0;jenkins-hbase4:44621] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-07 13:56:28,538 INFO [RS:0;jenkins-hbase4:44621] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44621,1691416588250-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:28,549 INFO [RS:0;jenkins-hbase4:44621] regionserver.Replication(203): jenkins-hbase4.apache.org,44621,1691416588250 started 2023-08-07 13:56:28,549 INFO [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,44621,1691416588250, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:44621, sessionid=0x101dfb6a1a10033 2023-08-07 13:56:28,549 DEBUG [RS:0;jenkins-hbase4:44621] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-07 13:56:28,549 DEBUG [RS:0;jenkins-hbase4:44621] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:28,549 DEBUG [RS:0;jenkins-hbase4:44621] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44621,1691416588250' 2023-08-07 13:56:28,549 DEBUG [RS:0;jenkins-hbase4:44621] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-08-07 13:56:28,550 DEBUG [RS:0;jenkins-hbase4:44621] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-08-07 13:56:28,550 DEBUG [RS:0;jenkins-hbase4:44621] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-07 13:56:28,550 DEBUG [RS:0;jenkins-hbase4:44621] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-07 13:56:28,550 DEBUG [RS:0;jenkins-hbase4:44621] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:28,550 DEBUG [RS:0;jenkins-hbase4:44621] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44621,1691416588250' 2023-08-07 13:56:28,550 DEBUG [RS:0;jenkins-hbase4:44621] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-08-07 13:56:28,550 DEBUG [RS:0;jenkins-hbase4:44621] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-08-07 13:56:28,551 DEBUG [RS:0;jenkins-hbase4:44621] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-07 13:56:28,551 INFO [RS:0;jenkins-hbase4:44621] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-07 13:56:28,551 INFO [RS:0;jenkins-hbase4:44621] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-07 13:56:28,613 DEBUG [jenkins-hbase4:35101] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-07 13:56:28,614 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,44621,1691416588250, state=OPENING 2023-08-07 13:56:28,616 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-07 13:56:28,616 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-07 13:56:28,616 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,44621,1691416588250}] 2023-08-07 13:56:28,651 INFO [RS:0;jenkins-hbase4:44621] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:56:28,653 INFO [RS:0;jenkins-hbase4:44621] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44621%2C1691416588250.jenkins-hbase4.apache.org%2C44621%2C1691416588250.regiongroup-0, suffix=, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,44621,1691416588250, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs, maxLogs=10 2023-08-07 13:56:28,668 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:56:28,669 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK] 2023-08-07 13:56:28,669 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK] 2023-08-07 13:56:28,671 INFO [RS:0;jenkins-hbase4:44621] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,44621,1691416588250/jenkins-hbase4.apache.org%2C44621%2C1691416588250.jenkins-hbase4.apache.org%2C44621%2C1691416588250.regiongroup-0.1691416588653 2023-08-07 13:56:28,671 DEBUG [RS:0;jenkins-hbase4:44621] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK], DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK], DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK]] 2023-08-07 13:56:28,771 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:28,772 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 13:56:28,773 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48472, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 13:56:28,777 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-07 13:56:28,777 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:56:28,777 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:56:28,777 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:56:28,778 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44621%2C1691416588250.meta, suffix=.meta, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,44621,1691416588250, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs, maxLogs=10 2023-08-07 13:56:28,791 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK] 2023-08-07 13:56:28,791 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK] 2023-08-07 13:56:28,791 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK] 2023-08-07 13:56:28,794 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,44621,1691416588250/jenkins-hbase4.apache.org%2C44621%2C1691416588250.meta.1691416588779.meta 2023-08-07 13:56:28,794 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK], DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK], DatanodeInfoWithStorage[127.0.0.1:35779,DS-199fa256-8376-4ed4-8432-68ef01676f76,DISK]] 2023-08-07 13:56:28,794 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:56:28,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-07 13:56:28,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-07 13:56:28,795 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-07 13:56:28,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-07 13:56:28,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:56:28,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-07 13:56:28,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-07 13:56:28,797 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-07 13:56:28,798 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info 2023-08-07 13:56:28,798 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info 2023-08-07 13:56:28,798 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-07 13:56:28,804 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info/8766d5bf2c17410e8580146469a8662e 2023-08-07 13:56:28,808 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info/bf302993307c47d9bfad190ab0ffb485 2023-08-07 13:56:28,808 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:28,808 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-07 13:56:28,809 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/rep_barrier 2023-08-07 13:56:28,809 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/rep_barrier 2023-08-07 13:56:28,810 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-07 13:56:28,810 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:28,810 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-07 13:56:28,811 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/table 2023-08-07 13:56:28,811 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/table 2023-08-07 13:56:28,811 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-07 13:56:28,816 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/table/b6b6cd7163ac41e2b3c4c475d90c98bf 2023-08-07 13:56:28,816 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:28,817 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740 2023-08-07 13:56:28,818 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740 2023-08-07 13:56:28,820 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-07 13:56:28,821 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-07 13:56:28,822 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=35; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66228204, jitterRate=-0.013122856616973877}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-07 13:56:28,822 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-07 13:56:28,823 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=29, masterSystemTime=1691416588771 2023-08-07 13:56:28,827 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-07 13:56:28,827 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-07 13:56:28,828 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,44621,1691416588250, state=OPEN 2023-08-07 13:56:28,829 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-08-07 13:56:28,830 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-08-07 13:56:28,831 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-08-07 13:56:28,831 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,44621,1691416588250 in 214 msec 2023-08-07 13:56:28,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-08-07 13:56:28,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 371 msec 2023-08-07 13:56:30,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1552ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-08-07 13:56:30,726 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 13:56:31,511 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3055ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-08-07 13:56:32,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-08-07 13:56:32,964 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-07 13:56:32,967 DEBUG [hconnection-0x14e09201-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:56:32,969 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38520, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:56:32,976 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=3bc256268a4ab1eec2fd5f34b09fb147, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46513,1691416553082, regionLocation=jenkins-hbase4.apache.org,46513,1691416553082, openSeqNum=10 2023-08-07 13:56:32,976 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=77564e07be12a17717602214bc11ceb0, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46513,1691416553082, regionLocation=jenkins-hbase4.apache.org,46513,1691416553082, openSeqNum=10 2023-08-07 13:56:32,977 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=75110bd10fe307a72d13190ea044da7a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46513,1691416553082, regionLocation=jenkins-hbase4.apache.org,46513,1691416553082, openSeqNum=13 2023-08-07 13:56:32,977 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-07 13:56:32,977 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1691416652977 2023-08-07 13:56:32,977 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1691416712977 2023-08-07 13:56:32,977 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 13 msec 2023-08-07 13:56:32,994 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,46513,1691416553082 had 4 regions 2023-08-07 13:56:32,994 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35101,1691416588221-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:32,994 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35101,1691416588221-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:32,994 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35101,1691416588221-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:32,994 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:35101, period=300000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:32,994 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-07 13:56:32,994 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. is NOT online; state={3bc256268a4ab1eec2fd5f34b09fb147 state=OPEN, ts=1691416592976, server=jenkins-hbase4.apache.org,46513,1691416553082}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-08-07 13:56:32,996 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46513,1691416553082, splitWal=true, meta=true, isMeta: false 2023-08-07 13:56:32,999 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46513,1691416553082-splitting dir is empty, no logs to split. 2023-08-07 13:56:32,999 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,46513,1691416553082 WAL count=0, meta=false 2023-08-07 13:56:32,999 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,46513,1691416553082/hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147., unknown_server=jenkins-hbase4.apache.org,46513,1691416553082/t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0., unknown_server=jenkins-hbase4.apache.org,46513,1691416553082/t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:56:33,002 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,46513,1691416553082-splitting dir is empty, no logs to split. 2023-08-07 13:56:33,002 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,46513,1691416553082 WAL count=0, meta=false 2023-08-07 13:56:33,002 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46513,1691416553082 WAL splitting is done? wals=0, meta=false 2023-08-07 13:56:33,006 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=77564e07be12a17717602214bc11ceb0, ASSIGN}, {pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=75110bd10fe307a72d13190ea044da7a, ASSIGN}, {pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=3bc256268a4ab1eec2fd5f34b09fb147, ASSIGN}] 2023-08-07 13:56:33,007 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=3bc256268a4ab1eec2fd5f34b09fb147, ASSIGN 2023-08-07 13:56:33,007 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=77564e07be12a17717602214bc11ceb0, ASSIGN 2023-08-07 13:56:33,007 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=75110bd10fe307a72d13190ea044da7a, ASSIGN 2023-08-07 13:56:33,008 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=3bc256268a4ab1eec2fd5f34b09fb147, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-07 13:56:33,008 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=77564e07be12a17717602214bc11ceb0, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-07 13:56:33,008 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=75110bd10fe307a72d13190ea044da7a, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-08-07 13:56:33,158 DEBUG [jenkins-hbase4:35101] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-07 13:56:33,162 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=3bc256268a4ab1eec2fd5f34b09fb147, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:33,162 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=75110bd10fe307a72d13190ea044da7a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:33,162 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=77564e07be12a17717602214bc11ceb0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:33,163 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691416593162"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416593162"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416593162"}]},"ts":"1691416593162"} 2023-08-07 13:56:33,163 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416593162"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416593162"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416593162"}]},"ts":"1691416593162"} 2023-08-07 13:56:33,163 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416593162"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1691416593162"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1691416593162"}]},"ts":"1691416593162"} 2023-08-07 13:56:33,165 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 75110bd10fe307a72d13190ea044da7a, server=jenkins-hbase4.apache.org,44621,1691416588250}] 2023-08-07 13:56:33,166 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=30, state=RUNNABLE; OpenRegionProcedure 77564e07be12a17717602214bc11ceb0, server=jenkins-hbase4.apache.org,44621,1691416588250}] 2023-08-07 13:56:33,168 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=35, ppid=32, state=RUNNABLE; OpenRegionProcedure 3bc256268a4ab1eec2fd5f34b09fb147, server=jenkins-hbase4.apache.org,44621,1691416588250}] 2023-08-07 13:56:33,323 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:56:33,323 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-07 13:56:33,325 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44621%2C1691416588250.jenkins-hbase4.apache.org%2C44621%2C1691416588250.regiongroup-1, suffix=, logDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,44621,1691416588250, archiveDir=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs, maxLogs=10 2023-08-07 13:56:33,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:56:33,326 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 75110bd10fe307a72d13190ea044da7a, NAME => 't2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:56:33,327 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:56:33,327 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:56:33,327 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:56:33,327 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:56:33,330 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:56:33,331 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1 2023-08-07 13:56:33,331 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1 2023-08-07 13:56:33,331 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75110bd10fe307a72d13190ea044da7a columnFamilyName cf1 2023-08-07 13:56:33,342 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK] 2023-08-07 13:56:33,342 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK] 2023-08-07 13:56:33,342 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK] 2023-08-07 13:56:33,342 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1dbe1760723d4f519052b5fc9df408e5 2023-08-07 13:56:33,343 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1/1dbe1760723d4f519052b5fc9df408e5 2023-08-07 13:56:33,352 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/WALs/jenkins-hbase4.apache.org,44621,1691416588250/jenkins-hbase4.apache.org%2C44621%2C1691416588250.jenkins-hbase4.apache.org%2C44621%2C1691416588250.regiongroup-1.1691416593325 2023-08-07 13:56:33,353 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37685,DS-695e2bb5-5b8e-4828-8c3f-5911aa1e9265,DISK], DatanodeInfoWithStorage[127.0.0.1:32865,DS-489b8abe-c957-46e6-ace5-75708cf8539b,DISK], DatanodeInfoWithStorage[127.0.0.1:40529,DS-c5d9121b-56df-412f-a8b4-f33a8dc9060a,DISK]] 2023-08-07 13:56:33,353 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3bc256268a4ab1eec2fd5f34b09fb147, NAME => 'hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:56:33,353 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1/8a858ad05b094550b24538261ebbea4e 2023-08-07 13:56:33,353 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:56:33,353 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(310): Store=75110bd10fe307a72d13190ea044da7a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:33,353 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:56:33,353 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:56:33,353 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:56:33,353 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:56:33,354 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/norep 2023-08-07 13:56:33,355 DEBUG [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/norep 2023-08-07 13:56:33,355 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75110bd10fe307a72d13190ea044da7a columnFamilyName norep 2023-08-07 13:56:33,355 INFO [StoreOpener-75110bd10fe307a72d13190ea044da7a-1] regionserver.HStore(310): Store=75110bd10fe307a72d13190ea044da7a/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:33,357 INFO [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:56:33,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a 2023-08-07 13:56:33,358 DEBUG [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/info 2023-08-07 13:56:33,358 DEBUG [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/info 2023-08-07 13:56:33,358 INFO [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bc256268a4ab1eec2fd5f34b09fb147 columnFamilyName info 2023-08-07 13:56:33,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a 2023-08-07 13:56:33,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 13:56:33,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:56:33,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 75110bd10fe307a72d13190ea044da7a; next sequenceid=20; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61055731, jitterRate=-0.09019871056079865}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 13:56:33,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 75110bd10fe307a72d13190ea044da7a: 2023-08-07 13:56:33,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a., pid=33, masterSystemTime=1691416593319 2023-08-07 13:56:33,371 DEBUG [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/info/2db3ce6e71eb4d9a95061ddf2a1afa69 2023-08-07 13:56:33,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:56:33,371 INFO [StoreOpener-3bc256268a4ab1eec2fd5f34b09fb147-1] regionserver.HStore(310): Store=3bc256268a4ab1eec2fd5f34b09fb147/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:33,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:56:33,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:56:33,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 77564e07be12a17717602214bc11ceb0, NAME => 't1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.', STARTKEY => '', ENDKEY => ''} 2023-08-07 13:56:33,372 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=75110bd10fe307a72d13190ea044da7a, regionState=OPEN, openSeqNum=20, regionLocation=jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:33,372 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416593372"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416593372"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416593372"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416593372"}]},"ts":"1691416593372"} 2023-08-07 13:56:33,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 77564e07be12a17717602214bc11ceb0 2023-08-07 13:56:33,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-07 13:56:33,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 77564e07be12a17717602214bc11ceb0 2023-08-07 13:56:33,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 77564e07be12a17717602214bc11ceb0 2023-08-07 13:56:33,373 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:56:33,374 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 77564e07be12a17717602214bc11ceb0 2023-08-07 13:56:33,374 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:56:33,375 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1 2023-08-07 13:56:33,375 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1 2023-08-07 13:56:33,376 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 77564e07be12a17717602214bc11ceb0 columnFamilyName cf1 2023-08-07 13:56:33,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-08-07 13:56:33,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 75110bd10fe307a72d13190ea044da7a, server=jenkins-hbase4.apache.org,44621,1691416588250 in 209 msec 2023-08-07 13:56:33,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=75110bd10fe307a72d13190ea044da7a, ASSIGN in 372 msec 2023-08-07 13:56:33,378 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:56:33,379 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3bc256268a4ab1eec2fd5f34b09fb147; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59494721, jitterRate=-0.11345957219600677}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-07 13:56:33,379 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3bc256268a4ab1eec2fd5f34b09fb147: 2023-08-07 13:56:33,380 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147., pid=35, masterSystemTime=1691416593319 2023-08-07 13:56:33,382 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:56:33,382 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:56:33,382 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=3bc256268a4ab1eec2fd5f34b09fb147, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:33,382 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1691416593382"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416593382"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416593382"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416593382"}]},"ts":"1691416593382"} 2023-08-07 13:56:33,386 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=35, resume processing ppid=32 2023-08-07 13:56:33,386 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=32, state=SUCCESS; OpenRegionProcedure 3bc256268a4ab1eec2fd5f34b09fb147, server=jenkins-hbase4.apache.org,44621,1691416588250 in 217 msec 2023-08-07 13:56:33,387 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=3bc256268a4ab1eec2fd5f34b09fb147, ASSIGN in 382 msec 2023-08-07 13:56:33,391 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 681f5294d01f4af78e7bb7068cc0d77e 2023-08-07 13:56:33,392 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1/681f5294d01f4af78e7bb7068cc0d77e 2023-08-07 13:56:33,398 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(539): loaded hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1/ed63a5fa2177464491100ded6dbb022b 2023-08-07 13:56:33,399 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(310): Store=77564e07be12a17717602214bc11ceb0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:33,399 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 77564e07be12a17717602214bc11ceb0 2023-08-07 13:56:33,400 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/norep 2023-08-07 13:56:33,400 DEBUG [StoreOpener-77564e07be12a17717602214bc11ceb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/norep 2023-08-07 13:56:33,400 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 77564e07be12a17717602214bc11ceb0 columnFamilyName norep 2023-08-07 13:56:33,401 INFO [StoreOpener-77564e07be12a17717602214bc11ceb0-1] regionserver.HStore(310): Store=77564e07be12a17717602214bc11ceb0/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-07 13:56:33,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0 2023-08-07 13:56:33,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0 2023-08-07 13:56:33,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-07 13:56:33,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 77564e07be12a17717602214bc11ceb0 2023-08-07 13:56:33,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 77564e07be12a17717602214bc11ceb0; next sequenceid=17; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61023845, jitterRate=-0.09067384898662567}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-07 13:56:33,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 77564e07be12a17717602214bc11ceb0: 2023-08-07 13:56:33,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0., pid=34, masterSystemTime=1691416593319 2023-08-07 13:56:33,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:56:33,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:56:33,410 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=77564e07be12a17717602214bc11ceb0, regionState=OPEN, openSeqNum=17, regionLocation=jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:33,410 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1691416593410"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1691416593410"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1691416593410"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1691416593410"}]},"ts":"1691416593410"} 2023-08-07 13:56:33,413 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=30 2023-08-07 13:56:33,413 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=30, state=SUCCESS; OpenRegionProcedure 77564e07be12a17717602214bc11ceb0, server=jenkins-hbase4.apache.org,44621,1691416588250 in 245 msec 2023-08-07 13:56:33,415 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-08-07 13:56:33,415 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,46513,1691416553082 after splitting done 2023-08-07 13:56:33,415 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=77564e07be12a17717602214bc11ceb0, ASSIGN in 409 msec 2023-08-07 13:56:33,415 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,46513,1691416553082 from processing; numProcessing=0 2023-08-07 13:56:33,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,46513,1691416553082, splitWal=true, meta=true in 5.0040 sec 2023-08-07 13:56:33,549 INFO [jenkins-hbase4:44621Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:56:33,995 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-08-07 13:56:34,010 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-08-07 13:56:34,013 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-08-07 13:56:34,013 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.731sec 2023-08-07 13:56:34,013 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-07 13:56:34,013 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-07 13:56:34,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-07 13:56:34,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35101,1691416588221-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-07 13:56:34,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35101,1691416588221-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-07 13:56:34,017 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-07 13:56:34,077 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(139): Connect 0x66bd5d8a to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:34,081 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4b851169, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:34,083 DEBUG [hconnection-0x616af9f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:56:34,084 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38536, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:56:34,086 INFO [Listener at localhost/41247] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-08-07 13:56:34,087 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x66bd5d8a to 127.0.0.1:52488 2023-08-07 13:56:34,087 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:34,088 INFO [Listener at localhost/41247] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:35101 after: jenkins-hbase4.apache.org:35101 2023-08-07 13:56:34,088 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(139): Connect 0x0d4de440 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:34,092 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3e94d821, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:34,514 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-08-07 13:56:34,515 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-07 13:56:34,515 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-07 13:56:34,515 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-08-07 13:56:34,594 DEBUG [hconnection-0x33174f81-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:56:34,596 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38552, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-07 13:56:34,615 INFO [Listener at localhost/41247] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1691416594615 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:56:34,619 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): syncupReplication16914165946150x0, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:56:34,623 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1691416594615-0x101dfb6a1a10038 connected Start Replication Server start 2023-08-07 13:56:34,624 INFO [Listener at localhost/41247] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-08-07 13:56:34,624 INFO [Listener at localhost/41247] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-08-07 13:56:34,638 INFO [Listener at localhost/41247] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-08-07 13:56:34,639 DEBUG [Listener at localhost/41247] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-08-07 13:56:34,649 INFO [Listener at localhost/41247] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-08-07 13:56:34,651 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,34615,1691416579859] other RSs: [] 2023-08-07 13:56:34,670 DEBUG [Listener at localhost/41247.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x077ff74f to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:34,674 DEBUG [Listener at localhost/41247.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@28b81041, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:34,674 INFO [Listener at localhost/41247.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:56:34,677 DEBUG [Listener at localhost/41247.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:56:34,678 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101dfb6a1a1003a connected 2023-08-07 13:56:34,678 INFO [Listener at localhost/41247.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=4d10afbc-00b5-4a8d-bcc6-ae0771b33a8c to cluster=bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:56:37,528 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,34615,1691416579859/1's WALs to 1691416594624.syncuptool.replication.org,1234,1 2023-08-07 13:56:37,530 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1.1691416585222 with data PBUF\x08\xB7\x06 2023-08-07 13:56:37,531 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0.1691416580708 with data PBUF\x08\xD8\x05 2023-08-07 13:56:37,532 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,34615,1691416579859/1's WALs to 1691416594624.syncuptool.replication.org,1234,1 2023-08-07 13:56:38,549 INFO [jenkins-hbase4:44621Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:56:38,830 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 13:56:39,534 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,34615,1691416579859] 2023-08-07 13:56:39,534 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,34615,1691416579859] 2023-08-07 13:56:39,545 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,34615,1691416579859] 2023-08-07 13:56:39,545 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,34615,1691416579859, ReplicationSource: 1, currentBandwidth=0 2023-08-07 13:56:39,545 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:39,565 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859] zookeeper.ReadOnlyZKClient(139): Connect 0x098f7309 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:39,571 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@660c0e62, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:39,571 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52488 2023-08-07 13:56:39,573 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-07 13:56:39,574 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101dfb6a1a1003c connected 2023-08-07 13:56:39,575 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,34615,1691416579859 (queues=2) is replicating from cluster=4d10afbc-00b5-4a8d-bcc6-ae0771b33a8c to cluster=bde33cad-f567-42f2-a3e0-63dd9997a2f5 2023-08-07 13:56:39,575 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1 2023-08-07 13:56:39,578 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,34615,1691416579859, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-07 13:56:39,578 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0 2023-08-07 13:56:39,579 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1 2023-08-07 13:56:39,580 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,34615,1691416579859, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-08-07 13:56:39,581 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0 2023-08-07 13:56:39,585 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0.1691416580708 2023-08-07 13:56:39,586 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-07 13:56:39,586 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,34615,1691416579859 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-07 13:56:39,586 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:39,590 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1.1691416585222 2023-08-07 13:56:39,590 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSourceWALReader(162): Read 300 WAL entries eligible for replication 2023-08-07 13:56:39,590 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-08-07 13:56:39,592 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,34615,1691416579859 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/112=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/113=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/114=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/115=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/116=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/117=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/118=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/119=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/120=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/121=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/122=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/123=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/124=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/125=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/126=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/127=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/128=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/129=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/130=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/131=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/132=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/133=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/134=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/135=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/136=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/137=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/138=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/139=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/140=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/141=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/142=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/143=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/144=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/145=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/146=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/147=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/148=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/149=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/150=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/151=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/152=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/153=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/154=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/155=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/156=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/157=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/158=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/159=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/160=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/161=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/162=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/163=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/164=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/165=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/166=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/167=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/168=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/169=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/170=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/171=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/172=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/173=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/174=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/175=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/176=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/177=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/178=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/179=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/180=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/181=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/182=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/183=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/184=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/185=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/186=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/187=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/188=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/189=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/190=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/191=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/192=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/193=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/194=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/195=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/196=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/197=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/198=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/199=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/200=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/201=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/202=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/203=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/204=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/205=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/206=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/207=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/208=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/209=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/210=[#edits: 1 = ],208}, {t1_syncup/e1cfded3452bdf9174e1f916e4d8e3f3/211=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/212=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/213=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/214=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/215=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/216=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/217=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/218=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/219=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/220=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/221=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/222=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/223=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/224=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/225=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/226=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/227=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/228=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/229=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/230=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/231=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/232=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/233=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/234=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/235=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/236=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/237=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/238=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/239=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/240=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/241=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/242=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/243=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/244=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/245=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/246=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/247=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/248=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/249=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/250=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/251=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/252=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/253=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/254=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/255=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/256=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/257=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/258=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/259=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/260=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/261=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/262=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/263=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/264=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/265=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/266=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/267=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/268=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/269=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/270=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/271=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/272=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/273=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/274=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/275=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/276=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/277=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/278=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/279=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/280=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/281=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/282=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/283=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/284=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/285=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/286=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/287=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/288=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/289=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/290=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/291=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/292=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/293=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/294=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/295=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/296=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/297=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/298=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/299=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/300=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/301=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/302=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/303=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/304=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/305=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/306=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/307=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/308=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/309=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/310=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/311=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/312=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/313=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/314=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/315=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/316=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/317=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/318=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/319=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/320=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/321=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/322=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/323=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/324=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/325=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/326=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/327=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/328=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/329=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/330=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/331=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/332=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/333=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/334=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/335=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/336=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/337=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/338=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/339=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/340=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/341=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/342=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/343=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/344=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/345=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/346=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/347=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/348=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/349=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/350=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/351=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/352=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/353=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/354=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/355=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/356=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/357=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/358=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/359=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/360=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/361=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/362=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/363=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/364=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/365=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/366=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/367=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/368=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/369=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/370=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/371=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/372=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/373=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/374=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/375=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/376=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/377=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/378=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/379=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/380=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/381=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/382=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/383=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/384=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/385=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/386=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/387=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/388=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/389=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/390=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/391=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/392=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/393=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/394=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/395=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/396=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/397=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/398=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/399=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/400=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/401=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/402=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/403=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/404=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/405=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/406=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/407=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/408=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/409=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/410=[#edits: 1 = ],208}, {t2_syncup/baa9fe21942fd12dea5caf4ddaa33efe/411=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:39335/user/jenkins/test-data/3e850cfa-897f-7dfc-ea74-7eb7e48cd99a/oldWALs/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1.1691416585222, lastWalPosition=34707, nbRowKeys=300, nbHFiles=0, heapSize=62400, lastSeqIds={}, endOfFile=true,usedBufferSize=62400] 2023-08-07 13:56:39,597 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-07 13:56:39,600 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38568, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-07 13:56:39,604 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44621] regionserver.ReplicationSink(263): Started replicating mutations. 2023-08-07 13:56:39,604 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44621] zookeeper.ReadOnlyZKClient(139): Connect 0x6fc006a5 to 127.0.0.1:52488 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-08-07 13:56:39,608 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44621] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@6107e52d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-07 13:56:39,610 DEBUG [hconnection-0x65386dd5-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-07 13:56:39,611 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38574, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=ClientService 2023-08-07 13:56:39,632 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44621] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-08-07 13:56:39,633 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1-jenkins-hbase4.apache.org,34615,1691416579859] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,34615,1691416579859] 2023-08-07 13:56:39,634 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1.1691416585222] 2023-08-07 13:56:39,641 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,34615,1691416579859 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-08-07 13:56:39,641 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:39,686 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:39,687 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1-jenkins-hbase4.apache.org,34615,1691416579859] zookeeper.ZKUtil(162): syncupReplication1691416594615-0x101dfb6a1a10038, quorum=127.0.0.1:52488, baseZNode=/1 Set watcher on existing znode=/1/replication/rs/1691416594624.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,34615,1691416579859/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0.1691416580708 2023-08-07 13:56:39,689 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691416594615-0x101dfb6a1a10038, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1691416594624.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,34615,1691416579859/jenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0.1691416580708 2023-08-07 13:56:39,689 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691416594615-0x101dfb6a1a10038, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/replication/rs/1691416594624.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:39,689 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-0,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,34615,1691416579859 with the following stats: Total replicated edits: 300, current progress: 2023-08-07 13:56:39,689 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691416594615-0x101dfb6a1a10038, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1691416594624.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:39,789 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,34615,1691416579859 2023-08-07 13:56:39,790 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1-jenkins-hbase4.apache.org,34615,1691416579859] zookeeper.ZKUtil(243): syncupReplication1691416594615-0x101dfb6a1a10038, quorum=127.0.0.1:52488, baseZNode=/1 Unable to list children of znode /1/replication/rs/1691416594624.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,34615,1691416579859 because node does not exist (not an error) 2023-08-07 13:56:39,790 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859.replicationSource.shipperjenkins-hbase4.apache.org%2C34615%2C1691416579859.jenkins-hbase4.apache.org%2C34615%2C1691416579859.regiongroup-1,1-jenkins-hbase4.apache.org,34615,1691416579859] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,34615,1691416579859 with the following stats: Total replicated edits: 300, current progress: 2023-08-07 13:56:43,549 INFO [jenkins-hbase4:44621Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-08-07 13:56:43,549 INFO [jenkins-hbase4:44621Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 300 2023-08-07 13:56:44,651 INFO [Listener at localhost/41247] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-08-07 13:56:44,755 DEBUG [Listener at localhost/41247.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a1003a, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:44,755 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x077ff74f to 127.0.0.1:52488 2023-08-07 13:56:44,756 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:44,958 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691416594615-0x101dfb6a1a10038, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:44,959 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): syncupReplication1691416594615-0x101dfb6a1a10038, quorum=127.0.0.1:52488, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:44,966 INFO [Listener at localhost/41247] replication.TestReplicationSyncUpTool(280): SyncUpAfterPut succeeded at retry = 0 2023-08-07 13:56:44,966 INFO [Listener at localhost/41247] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-07 13:56:44,966 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0d4de440 to 127.0.0.1:52488 2023-08-07 13:56:44,966 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:44,967 DEBUG [Listener at localhost/41247] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-07 13:56:44,967 DEBUG [Listener at localhost/41247] util.JVMClusterUtil(257): Found active master hash=1176323231, stopped=false 2023-08-07 13:56:44,967 INFO [Listener at localhost/41247] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,35101,1691416588221 2023-08-07 13:56:44,970 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:44621-0x101dfb6a1a10033, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-07 13:56:44,970 INFO [Listener at localhost/41247] procedure2.ProcedureExecutor(629): Stopping 2023-08-07 13:56:44,970 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-08-07 13:56:44,970 DEBUG [Listener at localhost/41247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x34afbec1 to 127.0.0.1:52488 2023-08-07 13:56:44,971 DEBUG [Listener at localhost/41247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:44,972 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:56:44,973 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44621-0x101dfb6a1a10033, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:56:44,973 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-08-07 13:56:44,972 INFO [Listener at localhost/41247] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,44621,1691416588250' ***** 2023-08-07 13:56:44,973 INFO [Listener at localhost/41247] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-07 13:56:44,974 INFO [RS:0;jenkins-hbase4:44621] regionserver.HeapMemoryManager(220): Stopping 2023-08-07 13:56:44,974 INFO [RS:0;jenkins-hbase4:44621] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-07 13:56:44,974 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-07 13:56:44,974 INFO [RS:0;jenkins-hbase4:44621] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-07 13:56:44,975 INFO [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(3305): Received CLOSE for 77564e07be12a17717602214bc11ceb0 2023-08-07 13:56:44,975 INFO [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(3305): Received CLOSE for 75110bd10fe307a72d13190ea044da7a 2023-08-07 13:56:44,975 INFO [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(3305): Received CLOSE for 3bc256268a4ab1eec2fd5f34b09fb147 2023-08-07 13:56:44,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 77564e07be12a17717602214bc11ceb0, disabling compactions & flushes 2023-08-07 13:56:44,976 INFO [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:44,976 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:56:44,976 DEBUG [RS:0;jenkins-hbase4:44621] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x283715be to 127.0.0.1:52488 2023-08-07 13:56:44,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:56:44,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. after waiting 0 ms 2023-08-07 13:56:44,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:56:44,976 DEBUG [RS:0;jenkins-hbase4:44621] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:44,977 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 77564e07be12a17717602214bc11ceb0 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-08-07 13:56:44,977 INFO [RS:0;jenkins-hbase4:44621] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-07 13:56:44,977 INFO [RS:0;jenkins-hbase4:44621] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-07 13:56:44,977 INFO [RS:0;jenkins-hbase4:44621] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-07 13:56:44,977 INFO [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-07 13:56:44,977 INFO [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-08-07 13:56:44,977 DEBUG [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, 77564e07be12a17717602214bc11ceb0=t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0., 75110bd10fe307a72d13190ea044da7a=t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a., 3bc256268a4ab1eec2fd5f34b09fb147=hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147.} 2023-08-07 13:56:44,977 DEBUG [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(1504): Waiting on 1588230740, 3bc256268a4ab1eec2fd5f34b09fb147, 75110bd10fe307a72d13190ea044da7a, 77564e07be12a17717602214bc11ceb0 2023-08-07 13:56:44,978 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-07 13:56:44,978 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-07 13:56:44,978 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-07 13:56:44,978 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-07 13:56:44,978 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-07 13:56:44,978 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-08-07 13:56:45,004 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=20 (bloomFilter=true), to=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/.tmp/cf1/51987945347a4c7b94f322d9d11b8cc5 2023-08-07 13:56:45,009 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=43 (bloomFilter=false), to=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/.tmp/info/8dec95d43165449fb892e78345a311b5 2023-08-07 13:56:45,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/.tmp/cf1/51987945347a4c7b94f322d9d11b8cc5 as hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1/51987945347a4c7b94f322d9d11b8cc5 2023-08-07 13:56:45,015 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/.tmp/info/8dec95d43165449fb892e78345a311b5 as hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info/8dec95d43165449fb892e78345a311b5 2023-08-07 13:56:45,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/cf1/51987945347a4c7b94f322d9d11b8cc5, entries=100, sequenceid=20, filesize=8.3 K 2023-08-07 13:56:45,021 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 77564e07be12a17717602214bc11ceb0 in 45ms, sequenceid=20, compaction requested=true 2023-08-07 13:56:45,031 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/info/8dec95d43165449fb892e78345a311b5, entries=24, sequenceid=43, filesize=7.4 K 2023-08-07 13:56:45,035 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 57ms, sequenceid=43, compaction requested=true 2023-08-07 13:56:45,038 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:56:45,040 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t1_syncup/77564e07be12a17717602214bc11ceb0/recovered.edits/23.seqid, newMaxSeqId=23, maxSeqId=16 2023-08-07 13:56:45,050 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:56:45,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 77564e07be12a17717602214bc11ceb0: 2023-08-07 13:56:45,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1691416547451.77564e07be12a17717602214bc11ceb0. 2023-08-07 13:56:45,051 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 75110bd10fe307a72d13190ea044da7a, disabling compactions & flushes 2023-08-07 13:56:45,051 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:56:45,051 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:56:45,051 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. after waiting 0 ms 2023-08-07 13:56:45,051 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:56:45,051 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 75110bd10fe307a72d13190ea044da7a 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-08-07 13:56:45,055 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/meta/1588230740/recovered.edits/46.seqid, newMaxSeqId=46, maxSeqId=34 2023-08-07 13:56:45,055 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-07 13:56:45,057 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-07 13:56:45,057 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-07 13:56:45,057 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-07 13:56:45,070 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=23 (bloomFilter=true), to=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/.tmp/cf1/03abb479901748b8807281d96f96fa55 2023-08-07 13:56:45,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/.tmp/cf1/03abb479901748b8807281d96f96fa55 as hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1/03abb479901748b8807281d96f96fa55 2023-08-07 13:56:45,088 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/cf1/03abb479901748b8807281d96f96fa55, entries=200, sequenceid=23, filesize=12.2 K 2023-08-07 13:56:45,089 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 75110bd10fe307a72d13190ea044da7a in 38ms, sequenceid=23, compaction requested=true 2023-08-07 13:56:45,100 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/default/t2_syncup/75110bd10fe307a72d13190ea044da7a/recovered.edits/26.seqid, newMaxSeqId=26, maxSeqId=19 2023-08-07 13:56:45,102 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:56:45,102 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 75110bd10fe307a72d13190ea044da7a: 2023-08-07 13:56:45,102 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1691416548067.75110bd10fe307a72d13190ea044da7a. 2023-08-07 13:56:45,103 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3bc256268a4ab1eec2fd5f34b09fb147, disabling compactions & flushes 2023-08-07 13:56:45,103 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:56:45,103 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:56:45,103 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. after waiting 0 ms 2023-08-07 13:56:45,103 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:56:45,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/data/hbase/namespace/3bc256268a4ab1eec2fd5f34b09fb147/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=12 2023-08-07 13:56:45,113 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:56:45,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3bc256268a4ab1eec2fd5f34b09fb147: 2023-08-07 13:56:45,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1691416545470.3bc256268a4ab1eec2fd5f34b09fb147. 2023-08-07 13:56:45,177 INFO [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,44621,1691416588250; all regions closed. 2023-08-07 13:56:45,182 DEBUG [RS:0;jenkins-hbase4:44621] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs 2023-08-07 13:56:45,182 INFO [RS:0;jenkins-hbase4:44621] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44621%2C1691416588250.meta:.meta(num 1691416588779) 2023-08-07 13:56:45,190 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-07 13:56:45,191 DEBUG [RS:0;jenkins-hbase4:44621] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs 2023-08-07 13:56:45,191 INFO [RS:0;jenkins-hbase4:44621] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44621%2C1691416588250.jenkins-hbase4.apache.org%2C44621%2C1691416588250.regiongroup-1:(num 1691416593325) 2023-08-07 13:56:45,202 DEBUG [RS:0;jenkins-hbase4:44621] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/oldWALs 2023-08-07 13:56:45,203 INFO [RS:0;jenkins-hbase4:44621] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44621%2C1691416588250.jenkins-hbase4.apache.org%2C44621%2C1691416588250.regiongroup-0:(num 1691416588653) 2023-08-07 13:56:45,203 DEBUG [RS:0;jenkins-hbase4:44621] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:45,203 INFO [RS:0;jenkins-hbase4:44621] regionserver.LeaseManager(133): Closed leases 2023-08-07 13:56:45,203 INFO [RS:0;jenkins-hbase4:44621] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-07 13:56:45,203 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 13:56:45,203 DEBUG [RS:0;jenkins-hbase4:44621] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6fc006a5 to 127.0.0.1:52488 2023-08-07 13:56:45,204 DEBUG [RS:0;jenkins-hbase4:44621] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:45,205 INFO [RS:0;jenkins-hbase4:44621] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44621 2023-08-07 13:56:45,210 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:44621-0x101dfb6a1a10033, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,44621,1691416588250 2023-08-07 13:56:45,211 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:56:45,210 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a1003c, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:56:45,210 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101dfb6a1a10026, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:56:45,211 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-07 13:56:45,211 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:44621-0x101dfb6a1a10033, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-08-07 13:56:45,211 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-08-07 13:56:45,212 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,44621,1691416588250] 2023-08-07 13:56:45,212 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,44621,1691416588250; numProcessing=1 2023-08-07 13:56:45,213 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,44621,1691416588250 already deleted, retry=false 2023-08-07 13:56:45,213 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,44621,1691416588250 expired; onlineServers=0 2023-08-07 13:56:45,213 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,35101,1691416588221' ***** 2023-08-07 13:56:45,213 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-07 13:56:45,214 DEBUG [M:0;jenkins-hbase4:35101] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4222d884, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-07 13:56:45,214 INFO [M:0;jenkins-hbase4:35101] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,35101,1691416588221 2023-08-07 13:56:45,214 INFO [M:0;jenkins-hbase4:35101] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,35101,1691416588221; all regions closed. 2023-08-07 13:56:45,214 DEBUG [M:0;jenkins-hbase4:35101] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-07 13:56:45,214 DEBUG [M:0;jenkins-hbase4:35101] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-07 13:56:45,214 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-07 13:56:45,215 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416588455] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1691416588455,5,FailOnTimeoutGroup] 2023-08-07 13:56:45,214 DEBUG [M:0;jenkins-hbase4:35101] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-07 13:56:45,214 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416588455] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1691416588455,5,FailOnTimeoutGroup] 2023-08-07 13:56:45,216 INFO [M:0;jenkins-hbase4:35101] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-07 13:56:45,216 INFO [M:0;jenkins-hbase4:35101] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-07 13:56:45,216 INFO [M:0;jenkins-hbase4:35101] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-07 13:56:45,216 DEBUG [M:0;jenkins-hbase4:35101] master.HMaster(1512): Stopping service threads 2023-08-07 13:56:45,216 INFO [M:0;jenkins-hbase4:35101] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-07 13:56:45,217 ERROR [M:0;jenkins-hbase4:35101] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-07 13:56:45,217 INFO [M:0;jenkins-hbase4:35101] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-07 13:56:45,217 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-08-07 13:56:45,217 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-07 13:56:45,217 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-08-07 13:56:45,217 DEBUG [M:0;jenkins-hbase4:35101] zookeeper.ZKUtil(398): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-08-07 13:56:45,217 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-08-07 13:56:45,217 WARN [M:0;jenkins-hbase4:35101] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-07 13:56:45,218 INFO [M:0;jenkins-hbase4:35101] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-07 13:56:45,218 INFO [M:0;jenkins-hbase4:35101] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-07 13:56:45,219 DEBUG [M:0;jenkins-hbase4:35101] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-07 13:56:45,219 INFO [M:0;jenkins-hbase4:35101] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:56:45,219 DEBUG [M:0;jenkins-hbase4:35101] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:56:45,219 DEBUG [M:0;jenkins-hbase4:35101] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-07 13:56:45,219 DEBUG [M:0;jenkins-hbase4:35101] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:56:45,219 INFO [M:0;jenkins-hbase4:35101] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=26.42 KB heapSize=32.85 KB 2023-08-07 13:56:45,236 INFO [M:0;jenkins-hbase4:35101] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=26.42 KB at sequenceid=310 (bloomFilter=true), to=hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ce66d38f55a04d5992fbc95ddfbaa765 2023-08-07 13:56:45,242 DEBUG [M:0;jenkins-hbase4:35101] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ce66d38f55a04d5992fbc95ddfbaa765 as hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ce66d38f55a04d5992fbc95ddfbaa765 2023-08-07 13:56:45,247 INFO [M:0;jenkins-hbase4:35101] regionserver.HStore(1080): Added hdfs://localhost:45993/user/jenkins/test-data/702c4221-0458-9552-ca03-743b7d34d1e2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ce66d38f55a04d5992fbc95ddfbaa765, entries=9, sequenceid=310, filesize=5.5 K 2023-08-07 13:56:45,248 INFO [M:0;jenkins-hbase4:35101] regionserver.HRegion(2948): Finished flush of dataSize ~26.42 KB/27051, heapSize ~32.84 KB/33624, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 29ms, sequenceid=310, compaction requested=true 2023-08-07 13:56:45,250 INFO [M:0;jenkins-hbase4:35101] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-07 13:56:45,250 DEBUG [M:0;jenkins-hbase4:35101] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-07 13:56:45,255 INFO [M:0;jenkins-hbase4:35101] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-07 13:56:45,255 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-07 13:56:45,255 INFO [M:0;jenkins-hbase4:35101] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35101 2023-08-07 13:56:45,259 DEBUG [M:0;jenkins-hbase4:35101] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,35101,1691416588221 already deleted, retry=false 2023-08-07 13:56:45,407 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:45,407 INFO [M:0;jenkins-hbase4:35101] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,35101,1691416588221; zookeeper connection closed. 2023-08-07 13:56:45,407 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): master:35101-0x101dfb6a1a10032, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:45,507 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:44621-0x101dfb6a1a10033, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:45,507 INFO [RS:0;jenkins-hbase4:44621] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,44621,1691416588250; zookeeper connection closed. 2023-08-07 13:56:45,507 DEBUG [Listener at localhost/41247-EventThread] zookeeper.ZKWatcher(600): regionserver:44621-0x101dfb6a1a10033, quorum=127.0.0.1:52488, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-07 13:56:45,508 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@32dd5359] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@32dd5359 2023-08-07 13:56:45,508 INFO [Listener at localhost/41247] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-07 13:56:45,508 WARN [Listener at localhost/41247] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 13:56:45,514 INFO [Listener at localhost/41247] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 13:56:45,619 WARN [BP-1500435777-172.31.14.131-1691416543114 heartbeating to localhost/127.0.0.1:45993] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 13:56:45,620 WARN [BP-1500435777-172.31.14.131-1691416543114 heartbeating to localhost/127.0.0.1:45993] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1500435777-172.31.14.131-1691416543114 (Datanode Uuid 6efc2cb2-167f-4008-b6ed-09aaade58b38) service to localhost/127.0.0.1:45993 2023-08-07 13:56:45,622 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/cluster_4d37a1ca-6c9d-592f-955d-f460037903b1/dfs/data/data7/current/BP-1500435777-172.31.14.131-1691416543114] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 13:56:45,622 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/cluster_4d37a1ca-6c9d-592f-955d-f460037903b1/dfs/data/data8/current/BP-1500435777-172.31.14.131-1691416543114] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 13:56:45,626 WARN [Listener at localhost/41247] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 13:56:45,629 INFO [Listener at localhost/41247] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 13:56:45,735 WARN [BP-1500435777-172.31.14.131-1691416543114 heartbeating to localhost/127.0.0.1:45993] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 13:56:45,735 WARN [BP-1500435777-172.31.14.131-1691416543114 heartbeating to localhost/127.0.0.1:45993] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1500435777-172.31.14.131-1691416543114 (Datanode Uuid bc8e0135-89c1-470e-8efa-86ea0613a4e1) service to localhost/127.0.0.1:45993 2023-08-07 13:56:45,735 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/cluster_4d37a1ca-6c9d-592f-955d-f460037903b1/dfs/data/data5/current/BP-1500435777-172.31.14.131-1691416543114] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 13:56:45,736 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/cluster_4d37a1ca-6c9d-592f-955d-f460037903b1/dfs/data/data6/current/BP-1500435777-172.31.14.131-1691416543114] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 13:56:45,737 WARN [Listener at localhost/41247] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 13:56:45,742 INFO [Listener at localhost/41247] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 13:56:45,847 WARN [BP-1500435777-172.31.14.131-1691416543114 heartbeating to localhost/127.0.0.1:45993] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 13:56:45,847 WARN [BP-1500435777-172.31.14.131-1691416543114 heartbeating to localhost/127.0.0.1:45993] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1500435777-172.31.14.131-1691416543114 (Datanode Uuid cf80ca65-cded-43cf-8fd5-044c1877768f) service to localhost/127.0.0.1:45993 2023-08-07 13:56:45,847 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/cluster_4d37a1ca-6c9d-592f-955d-f460037903b1/dfs/data/data3/current/BP-1500435777-172.31.14.131-1691416543114] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 13:56:45,848 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/cluster_4d37a1ca-6c9d-592f-955d-f460037903b1/dfs/data/data4/current/BP-1500435777-172.31.14.131-1691416543114] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 13:56:45,849 WARN [Listener at localhost/41247] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 13:56:45,852 INFO [Listener at localhost/41247] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 13:56:45,958 WARN [BP-1500435777-172.31.14.131-1691416543114 heartbeating to localhost/127.0.0.1:45993] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 13:56:45,958 WARN [BP-1500435777-172.31.14.131-1691416543114 heartbeating to localhost/127.0.0.1:45993] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1500435777-172.31.14.131-1691416543114 (Datanode Uuid 9ff4caa8-b296-41e4-aa7c-8583e50bff2d) service to localhost/127.0.0.1:45993 2023-08-07 13:56:45,959 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/cluster_4d37a1ca-6c9d-592f-955d-f460037903b1/dfs/data/data1/current/BP-1500435777-172.31.14.131-1691416543114] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 13:56:45,959 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/808e1cd3-9cc2-3f64-abdd-796c9e749f7b/cluster_4d37a1ca-6c9d-592f-955d-f460037903b1/dfs/data/data2/current/BP-1500435777-172.31.14.131-1691416543114] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 13:56:45,991 INFO [Listener at localhost/41247] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 13:56:46,182 INFO [Listener at localhost/41247] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-07 13:56:46,182 INFO [Listener at localhost/41247] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-07 13:56:46,182 WARN [Listener at localhost/41247] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 13:56:46,184 INFO [Listener at localhost/41247] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 13:56:46,285 WARN [BP-881135318-172.31.14.131-1691416535853 heartbeating to localhost/127.0.0.1:39335] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 13:56:46,285 WARN [BP-881135318-172.31.14.131-1691416535853 heartbeating to localhost/127.0.0.1:39335] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-881135318-172.31.14.131-1691416535853 (Datanode Uuid 4164ac0a-8f80-434d-8841-c564236eb4dc) service to localhost/127.0.0.1:39335 2023-08-07 13:56:46,286 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/cluster_d148ea16-da3d-4a0f-a23f-762caf2bedda/dfs/data/data3/current/BP-881135318-172.31.14.131-1691416535853] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 13:56:46,286 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/cluster_d148ea16-da3d-4a0f-a23f-762caf2bedda/dfs/data/data4/current/BP-881135318-172.31.14.131-1691416535853] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 13:56:46,287 WARN [Listener at localhost/41247] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-07 13:56:46,289 INFO [Listener at localhost/41247] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 13:56:46,390 WARN [BP-881135318-172.31.14.131-1691416535853 heartbeating to localhost/127.0.0.1:39335] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-07 13:56:46,390 WARN [BP-881135318-172.31.14.131-1691416535853 heartbeating to localhost/127.0.0.1:39335] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-881135318-172.31.14.131-1691416535853 (Datanode Uuid bd1d5e30-8de9-4355-b26e-64688c9d6d85) service to localhost/127.0.0.1:39335 2023-08-07 13:56:46,391 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/cluster_d148ea16-da3d-4a0f-a23f-762caf2bedda/dfs/data/data1/current/BP-881135318-172.31.14.131-1691416535853] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 13:56:46,391 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/61b46d92-0891-003c-4e1f-ae776d40b0ff/cluster_d148ea16-da3d-4a0f-a23f-762caf2bedda/dfs/data/data2/current/BP-881135318-172.31.14.131-1691416535853] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-07 13:56:46,399 INFO [Listener at localhost/41247] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-07 13:56:46,513 INFO [Listener at localhost/41247] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-07 13:56:46,538 INFO [Listener at localhost/41247] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-07 13:56:46,561 INFO [Listener at localhost/41247] hbase.ResourceChecker(175): after: replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL#testSyncUpTool Thread=186 (was 10) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-15 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x3258a3be-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-6-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (550718473) connection to localhost/127.0.0.1:39335 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:45993 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:39335 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x6f56592e sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1958204485.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-10 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-9 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-11 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/41247 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-16 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x713b6b67-SendThread(127.0.0.1:52488) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x3258a3be sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1958204485.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-6-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost:39335 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:45993 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-14 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x303db9a9-SendThread(127.0.0.1:52488) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x6f56592e-SendThread(127.0.0.1:52488) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x098f7309 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1958204485.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x098f7309-SendThread(127.0.0.1:52488) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost:45993 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:39335 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x713b6b67 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1958204485.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:45993 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-12 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x303db9a9-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x098f7309-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182-SendThread(127.0.0.1:52488) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost:45993 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x6f56592e-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@1cb810a4 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: IPC Client (550718473) connection to localhost/127.0.0.1:45993 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:45993 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (550718473) connection to localhost/127.0.0.1:45993 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (550718473) connection to localhost/127.0.0.1:39335 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x303db9a9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1958204485.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-13 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x3258a3be-SendThread(127.0.0.1:52488) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: IPC Client (550718473) connection to localhost/127.0.0.1:45993 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,46383,1691416540182-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,38873,1691416540136-SendThread(127.0.0.1:52488) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:39335 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:45993 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52488@0x713b6b67-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34615,1691416579859-SendThread(127.0.0.1:52488) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-16-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) - Thread LEAK? -, OpenFileDescriptor=661 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=227 (was 231), ProcessCount=171 (was 171), AvailableMemoryMB=1570 (was 2687)