2023-07-12 15:22:55,103 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b 2023-07-12 15:22:55,111 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/hadoop-log-dir so I do NOT create it in target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed 2023-07-12 15:22:55,112 WARN [main] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/hadoop-log-dir Erasing configuration value by system value. 2023-07-12 15:22:55,112 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/hadoop-tmp-dir so I do NOT create it in target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed 2023-07-12 15:22:55,112 WARN [main] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-12 15:22:55,112 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed 2023-07-12 15:22:55,125 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool timeout: 13 mins 2023-07-12 15:22:55,163 INFO [Time-limited test] hbase.ResourceChecker(147): before: replication.TestReplicationSyncUpTool#testSyncUpTool Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=165, ProcessCount=165, AvailableMemoryMB=8380 2023-07-12 15:22:55,176 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/hadoop-log-dir so I do NOT create it in target/test-data/d0013ab6-2589-f6f4-5693-e37e4426ecba 2023-07-12 15:22:55,176 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/hadoop-log-dir Erasing configuration value by system value. 2023-07-12 15:22:55,176 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/hadoop-tmp-dir so I do NOT create it in target/test-data/d0013ab6-2589-f6f4-5693-e37e4426ecba 2023-07-12 15:22:55,176 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-12 15:22:55,176 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d0013ab6-2589-f6f4-5693-e37e4426ecba 2023-07-12 15:22:55,183 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/hadoop-log-dir so I do NOT create it in target/test-data/80664b1d-5e5a-a0b5-e4fe-08fabea38332 2023-07-12 15:22:55,183 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/hadoop-log-dir Erasing configuration value by system value. 2023-07-12 15:22:55,184 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/hadoop-tmp-dir so I do NOT create it in target/test-data/80664b1d-5e5a-a0b5-e4fe-08fabea38332 2023-07-12 15:22:55,184 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-12 15:22:55,184 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/80664b1d-5e5a-a0b5-e4fe-08fabea38332 2023-07-12 15:22:55,187 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/cluster_fd1ef24c-43dc-8e79-0fe3-8873ef60ba76, deleteOnExit=true 2023-07-12 15:22:55,301 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/cluster_fd1ef24c-43dc-8e79-0fe3-8873ef60ba76/zookeeper_0, clientPort=64130, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/cluster_fd1ef24c-43dc-8e79-0fe3-8873ef60ba76/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/cluster_fd1ef24c-43dc-8e79-0fe3-8873ef60ba76/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-12 15:22:55,316 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=64130 2023-07-12 15:22:55,320 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=2, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-12 15:22:55,320 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-12 15:22:55,321 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/test.cache.data in system properties and HBase conf 2023-07-12 15:22:55,322 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/hadoop.tmp.dir in system properties and HBase conf 2023-07-12 15:22:55,322 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/hadoop.log.dir in system properties and HBase conf 2023-07-12 15:22:55,322 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-12 15:22:55,323 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-12 15:22:55,323 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-12 15:22:55,420 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-12 15:22:55,779 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-12 15:22:55,783 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-12 15:22:55,784 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-12 15:22:55,784 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-12 15:22:55,784 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-12 15:22:55,785 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-12 15:22:55,785 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-12 15:22:55,785 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-12 15:22:55,785 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-12 15:22:55,786 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-12 15:22:55,786 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/nfs.dump.dir in system properties and HBase conf 2023-07-12 15:22:55,786 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/java.io.tmpdir in system properties and HBase conf 2023-07-12 15:22:55,786 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-12 15:22:55,787 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-12 15:22:55,787 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-12 15:22:56,294 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-12 15:22:56,299 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-12 15:22:56,581 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-12 15:22:56,722 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-12 15:22:56,736 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 15:22:56,780 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 15:22:56,826 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/java.io.tmpdir/Jetty_localhost_35305_hdfs____yk3vr4/webapp 2023-07-12 15:22:56,995 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35305 2023-07-12 15:22:57,017 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-12 15:22:57,017 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-12 15:22:57,577 WARN [Listener at localhost/41947] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 15:22:57,677 WARN [Listener at localhost/41947] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-12 15:22:57,703 WARN [Listener at localhost/41947] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 15:22:57,711 INFO [Listener at localhost/41947] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 15:22:57,718 INFO [Listener at localhost/41947] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/java.io.tmpdir/Jetty_localhost_37873_datanode____.d03dux/webapp 2023-07-12 15:22:57,840 INFO [Listener at localhost/41947] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37873 2023-07-12 15:22:58,157 WARN [Listener at localhost/43077] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 15:22:58,172 WARN [Listener at localhost/43077] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-12 15:22:58,175 WARN [Listener at localhost/43077] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 15:22:58,177 INFO [Listener at localhost/43077] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 15:22:58,181 INFO [Listener at localhost/43077] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/java.io.tmpdir/Jetty_localhost_38533_datanode____88dgaf/webapp 2023-07-12 15:22:58,283 INFO [Listener at localhost/43077] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38533 2023-07-12 15:22:58,291 WARN [Listener at localhost/35683] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 15:22:58,669 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xec0a7a43a12e0ee5: Processing first storage report for DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770 from datanode ebf16695-f1f5-40c3-8513-9c1fecd94eff 2023-07-12 15:22:58,670 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xec0a7a43a12e0ee5: from storage DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770 node DatanodeRegistration(127.0.0.1:38179, datanodeUuid=ebf16695-f1f5-40c3-8513-9c1fecd94eff, infoPort=34557, infoSecurePort=0, ipcPort=43077, storageInfo=lv=-57;cid=testClusterID;nsid=81662820;c=1689175376370), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-07-12 15:22:58,671 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x488ee5379b9f6d7e: Processing first storage report for DS-999b06e9-279a-47df-944d-eabe177df1c6 from datanode d8ff5e8d-40a1-4929-b4c9-36d95a7996e3 2023-07-12 15:22:58,671 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x488ee5379b9f6d7e: from storage DS-999b06e9-279a-47df-944d-eabe177df1c6 node DatanodeRegistration(127.0.0.1:34197, datanodeUuid=d8ff5e8d-40a1-4929-b4c9-36d95a7996e3, infoPort=44839, infoSecurePort=0, ipcPort=35683, storageInfo=lv=-57;cid=testClusterID;nsid=81662820;c=1689175376370), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 15:22:58,671 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xec0a7a43a12e0ee5: Processing first storage report for DS-222d6e5f-c5fe-4f5f-8be9-38896b6965c7 from datanode ebf16695-f1f5-40c3-8513-9c1fecd94eff 2023-07-12 15:22:58,671 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xec0a7a43a12e0ee5: from storage DS-222d6e5f-c5fe-4f5f-8be9-38896b6965c7 node DatanodeRegistration(127.0.0.1:38179, datanodeUuid=ebf16695-f1f5-40c3-8513-9c1fecd94eff, infoPort=34557, infoSecurePort=0, ipcPort=43077, storageInfo=lv=-57;cid=testClusterID;nsid=81662820;c=1689175376370), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 15:22:58,672 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x488ee5379b9f6d7e: Processing first storage report for DS-d337faca-f833-4270-b6fd-779887fb0f29 from datanode d8ff5e8d-40a1-4929-b4c9-36d95a7996e3 2023-07-12 15:22:58,672 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x488ee5379b9f6d7e: from storage DS-d337faca-f833-4270-b6fd-779887fb0f29 node DatanodeRegistration(127.0.0.1:34197, datanodeUuid=d8ff5e8d-40a1-4929-b4c9-36d95a7996e3, infoPort=44839, infoSecurePort=0, ipcPort=35683, storageInfo=lv=-57;cid=testClusterID;nsid=81662820;c=1689175376370), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-12 15:22:58,726 DEBUG [Listener at localhost/35683] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b 2023-07-12 15:22:58,738 INFO [Listener at localhost/35683] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:22:58,741 INFO [Listener at localhost/35683] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:22:59,407 INFO [Listener at localhost/35683] util.FSUtils(471): Created version file at hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4 with version=8 2023-07-12 15:22:59,407 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/hbase-staging 2023-07-12 15:22:59,745 INFO [Listener at localhost/35683] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-12 15:23:00,221 INFO [Listener at localhost/35683] client.ConnectionUtils(127): master/jenkins-hbase5:0 server-side Connection retries=45 2023-07-12 15:23:00,251 INFO [Listener at localhost/35683] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:00,252 INFO [Listener at localhost/35683] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:00,252 INFO [Listener at localhost/35683] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:23:00,252 INFO [Listener at localhost/35683] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:00,252 INFO [Listener at localhost/35683] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:23:00,394 INFO [Listener at localhost/35683] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:23:00,466 DEBUG [Listener at localhost/35683] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-12 15:23:00,577 INFO [Listener at localhost/35683] ipc.NettyRpcServer(120): Bind to /172.31.10.131:45903 2023-07-12 15:23:00,588 INFO [Listener at localhost/35683] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:00,591 INFO [Listener at localhost/35683] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:00,617 INFO [Listener at localhost/35683] zookeeper.RecoverableZooKeeper(93): Process identifier=master:45903 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:00,668 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:459030x0, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:00,685 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:45903-0x101c5bd87ea0000 connected 2023-07-12 15:23:00,739 DEBUG [Listener at localhost/35683] zookeeper.ZKUtil(164): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-12 15:23:00,740 DEBUG [Listener at localhost/35683] zookeeper.ZKUtil(164): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-12 15:23:00,744 DEBUG [Listener at localhost/35683] zookeeper.ZKUtil(164): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-12 15:23:00,753 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45903 2023-07-12 15:23:00,755 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45903 2023-07-12 15:23:00,756 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45903 2023-07-12 15:23:00,756 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45903 2023-07-12 15:23:00,756 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45903 2023-07-12 15:23:00,762 INFO [Listener at localhost/35683] master.HMaster(444): hbase.rootdir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4, hbase.cluster.distributed=false 2023-07-12 15:23:00,830 INFO [Listener at localhost/35683] client.ConnectionUtils(127): regionserver/jenkins-hbase5:0 server-side Connection retries=45 2023-07-12 15:23:00,830 INFO [Listener at localhost/35683] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:00,831 INFO [Listener at localhost/35683] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:00,831 INFO [Listener at localhost/35683] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:23:00,831 INFO [Listener at localhost/35683] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:00,831 INFO [Listener at localhost/35683] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:23:00,836 INFO [Listener at localhost/35683] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:23:00,839 INFO [Listener at localhost/35683] ipc.NettyRpcServer(120): Bind to /172.31.10.131:35567 2023-07-12 15:23:00,841 INFO [Listener at localhost/35683] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 15:23:00,847 DEBUG [Listener at localhost/35683] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 15:23:00,848 INFO [Listener at localhost/35683] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:00,850 INFO [Listener at localhost/35683] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:00,852 INFO [Listener at localhost/35683] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35567 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:00,855 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:355670x0, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:00,856 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:35567-0x101c5bd87ea0001 connected 2023-07-12 15:23:00,856 DEBUG [Listener at localhost/35683] zookeeper.ZKUtil(164): regionserver:35567-0x101c5bd87ea0001, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-12 15:23:00,857 DEBUG [Listener at localhost/35683] zookeeper.ZKUtil(164): regionserver:35567-0x101c5bd87ea0001, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-12 15:23:00,858 DEBUG [Listener at localhost/35683] zookeeper.ZKUtil(164): regionserver:35567-0x101c5bd87ea0001, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-12 15:23:00,859 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35567 2023-07-12 15:23:00,859 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35567 2023-07-12 15:23:00,860 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35567 2023-07-12 15:23:00,860 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35567 2023-07-12 15:23:00,861 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35567 2023-07-12 15:23:00,873 INFO [Listener at localhost/35683] client.ConnectionUtils(127): regionserver/jenkins-hbase5:0 server-side Connection retries=45 2023-07-12 15:23:00,874 INFO [Listener at localhost/35683] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:00,874 INFO [Listener at localhost/35683] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:00,874 INFO [Listener at localhost/35683] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:23:00,874 INFO [Listener at localhost/35683] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:00,874 INFO [Listener at localhost/35683] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:23:00,875 INFO [Listener at localhost/35683] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:23:00,876 INFO [Listener at localhost/35683] ipc.NettyRpcServer(120): Bind to /172.31.10.131:36017 2023-07-12 15:23:00,876 INFO [Listener at localhost/35683] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 15:23:00,880 DEBUG [Listener at localhost/35683] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 15:23:00,880 INFO [Listener at localhost/35683] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:00,882 INFO [Listener at localhost/35683] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:00,884 INFO [Listener at localhost/35683] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36017 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:00,887 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:360170x0, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:00,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36017-0x101c5bd87ea0002 connected 2023-07-12 15:23:00,888 DEBUG [Listener at localhost/35683] zookeeper.ZKUtil(164): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-12 15:23:00,889 DEBUG [Listener at localhost/35683] zookeeper.ZKUtil(164): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-12 15:23:00,890 DEBUG [Listener at localhost/35683] zookeeper.ZKUtil(164): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-12 15:23:00,890 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36017 2023-07-12 15:23:00,890 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36017 2023-07-12 15:23:00,891 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36017 2023-07-12 15:23:00,891 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36017 2023-07-12 15:23:00,891 DEBUG [Listener at localhost/35683] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36017 2023-07-12 15:23:00,893 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase5.apache.org,45903,1689175379564 2023-07-12 15:23:00,902 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-12 15:23:00,903 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase5.apache.org,45903,1689175379564 2023-07-12 15:23:00,921 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:35567-0x101c5bd87ea0001, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-12 15:23:00,921 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-12 15:23:00,921 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-12 15:23:00,922 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:00,922 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-12 15:23:00,924 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase5.apache.org,45903,1689175379564 from backup master directory 2023-07-12 15:23:00,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-12 15:23:00,927 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase5.apache.org,45903,1689175379564 2023-07-12 15:23:00,927 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-12 15:23:00,928 WARN [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:23:00,928 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase5.apache.org,45903,1689175379564 2023-07-12 15:23:00,931 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-12 15:23:00,932 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-12 15:23:01,023 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/hbase.id with ID: 098cb589-347a-4654-bf7c-f2a1c06d1dde 2023-07-12 15:23:01,062 INFO [master/jenkins-hbase5:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:01,082 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:01,125 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x7c3827af to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:01,159 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f8a7acc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:01,184 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-12 15:23:01,186 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-12 15:23:01,206 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-07-12 15:23:01,206 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-07-12 15:23:01,208 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-12 15:23:01,212 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-12 15:23:01,214 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:01,249 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store-tmp 2023-07-12 15:23:01,283 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:01,284 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-12 15:23:01,284 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:01,284 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:01,284 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-12 15:23:01,284 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:01,284 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:01,284 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:23:01,286 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/WALs/jenkins-hbase5.apache.org,45903,1689175379564 2023-07-12 15:23:01,308 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C45903%2C1689175379564, suffix=, logDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/WALs/jenkins-hbase5.apache.org,45903,1689175379564, archiveDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/oldWALs, maxLogs=10 2023-07-12 15:23:01,362 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38179,DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770,DISK] 2023-07-12 15:23:01,362 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34197,DS-999b06e9-279a-47df-944d-eabe177df1c6,DISK] 2023-07-12 15:23:01,370 DEBUG [RS-EventLoopGroup-4-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-12 15:23:01,443 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/WALs/jenkins-hbase5.apache.org,45903,1689175379564/jenkins-hbase5.apache.org%2C45903%2C1689175379564.1689175381318 2023-07-12 15:23:01,444 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34197,DS-999b06e9-279a-47df-944d-eabe177df1c6,DISK], DatanodeInfoWithStorage[127.0.0.1:38179,DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770,DISK]] 2023-07-12 15:23:01,444 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:01,445 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:01,449 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:01,450 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:01,510 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:01,517 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-12 15:23:01,542 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-12 15:23:01,554 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:01,559 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:01,561 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:01,575 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:01,580 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:23:01,581 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72154405, jitterRate=0.0751844197511673}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:23:01,581 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:23:01,583 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-12 15:23:01,602 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-12 15:23:01,602 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-12 15:23:01,618 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:23:01,622 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 3 msec 2023-07-12 15:23:01,657 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 33 msec 2023-07-12 15:23:01,657 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-12 15:23:01,683 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-12 15:23:01,689 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-12 15:23:01,712 INFO [master/jenkins-hbase5:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-12 15:23:01,716 INFO [master/jenkins-hbase5:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-12 15:23:01,717 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-12 15:23:01,722 INFO [master/jenkins-hbase5:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-12 15:23:01,726 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-12 15:23:01,730 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:01,731 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-12 15:23:01,732 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-12 15:23:01,743 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-12 15:23:01,747 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:35567-0x101c5bd87ea0001, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-12 15:23:01,747 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-12 15:23:01,747 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-12 15:23:01,748 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:01,748 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase5.apache.org,45903,1689175379564, sessionid=0x101c5bd87ea0000, setting cluster-up flag (Was=false) 2023-07-12 15:23:01,763 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:01,769 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-12 15:23:01,770 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase5.apache.org,45903,1689175379564 2023-07-12 15:23:01,777 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:01,781 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-12 15:23:01,783 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase5.apache.org,45903,1689175379564 2023-07-12 15:23:01,786 WARN [master/jenkins-hbase5:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.hbase-snapshot/.tmp 2023-07-12 15:23:01,795 INFO [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(951): ClusterId : 098cb589-347a-4654-bf7c-f2a1c06d1dde 2023-07-12 15:23:01,795 INFO [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(951): ClusterId : 098cb589-347a-4654-bf7c-f2a1c06d1dde 2023-07-12 15:23:01,800 DEBUG [RS:1;jenkins-hbase5:36017] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 15:23:01,800 DEBUG [RS:0;jenkins-hbase5:35567] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 15:23:01,808 DEBUG [RS:0;jenkins-hbase5:35567] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 15:23:01,808 DEBUG [RS:1;jenkins-hbase5:36017] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 15:23:01,808 DEBUG [RS:0;jenkins-hbase5:35567] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 15:23:01,809 DEBUG [RS:1;jenkins-hbase5:36017] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 15:23:01,813 DEBUG [RS:0;jenkins-hbase5:35567] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 15:23:01,813 DEBUG [RS:1;jenkins-hbase5:36017] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 15:23:01,815 DEBUG [RS:0;jenkins-hbase5:35567] zookeeper.ReadOnlyZKClient(139): Connect 0x139497cc to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:01,815 DEBUG [RS:1;jenkins-hbase5:36017] zookeeper.ReadOnlyZKClient(139): Connect 0x711237df to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:01,828 DEBUG [RS:0;jenkins-hbase5:35567] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d537bce, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:01,828 DEBUG [RS:1;jenkins-hbase5:36017] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7fd5d78d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:01,828 DEBUG [RS:1;jenkins-hbase5:36017] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7bea5caf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase5.apache.org/172.31.10.131:0 2023-07-12 15:23:01,828 DEBUG [RS:0;jenkins-hbase5:35567] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@196cfcbc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase5.apache.org/172.31.10.131:0 2023-07-12 15:23:01,853 DEBUG [RS:0;jenkins-hbase5:35567] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase5:35567 2023-07-12 15:23:01,853 DEBUG [RS:1;jenkins-hbase5:36017] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase5:36017 2023-07-12 15:23:01,857 INFO [RS:0;jenkins-hbase5:35567] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 15:23:01,857 INFO [RS:1;jenkins-hbase5:36017] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 15:23:01,858 INFO [RS:1;jenkins-hbase5:36017] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 15:23:01,858 INFO [RS:0;jenkins-hbase5:35567] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 15:23:01,858 DEBUG [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 15:23:01,858 DEBUG [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 15:23:01,861 INFO [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase5.apache.org,45903,1689175379564 with isa=jenkins-hbase5.apache.org/172.31.10.131:35567, startcode=1689175380829 2023-07-12 15:23:01,861 INFO [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase5.apache.org,45903,1689175379564 with isa=jenkins-hbase5.apache.org/172.31.10.131:36017, startcode=1689175380873 2023-07-12 15:23:01,879 DEBUG [RS:1;jenkins-hbase5:36017] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 15:23:01,879 DEBUG [RS:0;jenkins-hbase5:35567] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 15:23:01,933 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-12 15:23:01,938 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:60259, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 15:23:01,938 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:41821, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 15:23:01,946 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:01,946 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:01,946 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:01,946 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:01,946 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase5:0, corePoolSize=10, maxPoolSize=10 2023-07-12 15:23:01,946 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:01,946 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:23:01,947 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:01,950 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45903] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-12 15:23:01,954 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689175411954 2023-07-12 15:23:01,957 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-12 15:23:01,960 DEBUG [PEWorker-2] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-12 15:23:01,960 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45903] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-12 15:23:01,960 INFO [PEWorker-2] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-12 15:23:01,967 INFO [PEWorker-2] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-12 15:23:01,969 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-12 15:23:01,975 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-12 15:23:01,976 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-12 15:23:01,976 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-12 15:23:01,976 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-12 15:23:01,977 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:01,978 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-12 15:23:01,979 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-12 15:23:01,980 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-12 15:23:01,981 DEBUG [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(2830): Master is not running yet 2023-07-12 15:23:01,981 DEBUG [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(2830): Master is not running yet 2023-07-12 15:23:01,982 WARN [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-12 15:23:01,982 WARN [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-12 15:23:01,982 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-12 15:23:01,982 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-12 15:23:01,984 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175381984,5,FailOnTimeoutGroup] 2023-07-12 15:23:01,985 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175381984,5,FailOnTimeoutGroup] 2023-07-12 15:23:01,985 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:01,985 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-12 15:23:01,986 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:01,986 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,001 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-12 15:23:02,002 INFO [PEWorker-2] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-12 15:23:02,003 INFO [PEWorker-2] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4 2023-07-12 15:23:02,019 DEBUG [PEWorker-2] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:02,022 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-12 15:23:02,024 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/info 2023-07-12 15:23:02,025 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-12 15:23:02,027 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:02,027 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-12 15:23:02,030 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:23:02,031 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-12 15:23:02,032 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:02,032 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-12 15:23:02,034 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/table 2023-07-12 15:23:02,034 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-12 15:23:02,035 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:02,037 DEBUG [PEWorker-2] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740 2023-07-12 15:23:02,038 DEBUG [PEWorker-2] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740 2023-07-12 15:23:02,042 DEBUG [PEWorker-2] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-12 15:23:02,044 DEBUG [PEWorker-2] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-12 15:23:02,048 DEBUG [PEWorker-2] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:23:02,049 INFO [PEWorker-2] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73743197, jitterRate=0.0988592654466629}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-12 15:23:02,049 DEBUG [PEWorker-2] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-12 15:23:02,049 DEBUG [PEWorker-2] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-12 15:23:02,049 INFO [PEWorker-2] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-12 15:23:02,049 DEBUG [PEWorker-2] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-12 15:23:02,049 DEBUG [PEWorker-2] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-12 15:23:02,049 DEBUG [PEWorker-2] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-12 15:23:02,050 INFO [PEWorker-2] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-12 15:23:02,051 DEBUG [PEWorker-2] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-12 15:23:02,055 DEBUG [PEWorker-2] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-12 15:23:02,055 INFO [PEWorker-2] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-12 15:23:02,063 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-12 15:23:02,075 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-12 15:23:02,077 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-12 15:23:02,083 INFO [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase5.apache.org,45903,1689175379564 with isa=jenkins-hbase5.apache.org/172.31.10.131:36017, startcode=1689175380873 2023-07-12 15:23:02,083 INFO [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase5.apache.org,45903,1689175379564 with isa=jenkins-hbase5.apache.org/172.31.10.131:35567, startcode=1689175380829 2023-07-12 15:23:02,088 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45903] master.ServerManager(394): Registering regionserver=jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:02,093 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45903] master.ServerManager(394): Registering regionserver=jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:02,094 DEBUG [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4 2023-07-12 15:23:02,094 DEBUG [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:41947 2023-07-12 15:23:02,094 DEBUG [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 15:23:02,095 DEBUG [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4 2023-07-12 15:23:02,095 DEBUG [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:41947 2023-07-12 15:23:02,095 DEBUG [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 15:23:02,099 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-12 15:23:02,100 DEBUG [RS:1;jenkins-hbase5:36017] zookeeper.ZKUtil(162): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:02,101 WARN [RS:1;jenkins-hbase5:36017] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:23:02,101 INFO [RS:1;jenkins-hbase5:36017] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:02,101 DEBUG [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1948): logDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:02,101 DEBUG [RS:0;jenkins-hbase5:35567] zookeeper.ZKUtil(162): regionserver:35567-0x101c5bd87ea0001, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:02,101 WARN [RS:0;jenkins-hbase5:35567] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:23:02,101 INFO [RS:0;jenkins-hbase5:35567] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:02,102 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase5.apache.org,35567,1689175380829] 2023-07-12 15:23:02,102 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase5.apache.org,36017,1689175380873] 2023-07-12 15:23:02,102 DEBUG [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(1948): logDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:02,111 DEBUG [RS:1;jenkins-hbase5:36017] zookeeper.ZKUtil(162): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:02,111 DEBUG [RS:0;jenkins-hbase5:35567] zookeeper.ZKUtil(162): regionserver:35567-0x101c5bd87ea0001, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:02,111 DEBUG [RS:1;jenkins-hbase5:36017] zookeeper.ZKUtil(162): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:02,112 DEBUG [RS:0;jenkins-hbase5:35567] zookeeper.ZKUtil(162): regionserver:35567-0x101c5bd87ea0001, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:02,122 DEBUG [RS:0;jenkins-hbase5:35567] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-12 15:23:02,122 DEBUG [RS:1;jenkins-hbase5:36017] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-12 15:23:02,132 INFO [RS:0;jenkins-hbase5:35567] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 15:23:02,132 INFO [RS:1;jenkins-hbase5:36017] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 15:23:02,152 INFO [RS:1;jenkins-hbase5:36017] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 15:23:02,152 INFO [RS:0;jenkins-hbase5:35567] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 15:23:02,156 INFO [RS:1;jenkins-hbase5:36017] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 15:23:02,156 INFO [RS:0;jenkins-hbase5:35567] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 15:23:02,156 INFO [RS:1;jenkins-hbase5:36017] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,156 INFO [RS:0;jenkins-hbase5:35567] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,158 INFO [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 15:23:02,158 INFO [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 15:23:02,167 INFO [RS:1;jenkins-hbase5:36017] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,167 INFO [RS:0;jenkins-hbase5:35567] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,167 DEBUG [RS:1;jenkins-hbase5:36017] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,167 DEBUG [RS:0;jenkins-hbase5:35567] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,168 DEBUG [RS:1;jenkins-hbase5:36017] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,168 DEBUG [RS:0;jenkins-hbase5:35567] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,168 DEBUG [RS:1;jenkins-hbase5:36017] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,168 DEBUG [RS:0;jenkins-hbase5:35567] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,168 DEBUG [RS:1;jenkins-hbase5:36017] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,168 DEBUG [RS:0;jenkins-hbase5:35567] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,168 DEBUG [RS:1;jenkins-hbase5:36017] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,168 DEBUG [RS:0;jenkins-hbase5:35567] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,168 DEBUG [RS:1;jenkins-hbase5:36017] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase5:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:23:02,168 DEBUG [RS:0;jenkins-hbase5:35567] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase5:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:23:02,169 DEBUG [RS:1;jenkins-hbase5:36017] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,169 DEBUG [RS:0;jenkins-hbase5:35567] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,169 DEBUG [RS:1;jenkins-hbase5:36017] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,169 DEBUG [RS:0;jenkins-hbase5:35567] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,169 DEBUG [RS:1;jenkins-hbase5:36017] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,169 DEBUG [RS:0;jenkins-hbase5:35567] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,169 DEBUG [RS:1;jenkins-hbase5:36017] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,169 DEBUG [RS:0;jenkins-hbase5:35567] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:02,171 INFO [RS:0;jenkins-hbase5:35567] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,171 INFO [RS:0;jenkins-hbase5:35567] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,171 INFO [RS:0;jenkins-hbase5:35567] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,171 INFO [RS:1;jenkins-hbase5:36017] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,171 INFO [RS:1;jenkins-hbase5:36017] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,171 INFO [RS:1;jenkins-hbase5:36017] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,190 INFO [RS:0;jenkins-hbase5:35567] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 15:23:02,190 INFO [RS:1;jenkins-hbase5:36017] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 15:23:02,193 INFO [RS:1;jenkins-hbase5:36017] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,36017,1689175380873-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,193 INFO [RS:0;jenkins-hbase5:35567] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,35567,1689175380829-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,211 INFO [RS:1;jenkins-hbase5:36017] regionserver.Replication(203): jenkins-hbase5.apache.org,36017,1689175380873 started 2023-07-12 15:23:02,211 INFO [RS:0;jenkins-hbase5:35567] regionserver.Replication(203): jenkins-hbase5.apache.org,35567,1689175380829 started 2023-07-12 15:23:02,211 INFO [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1637): Serving as jenkins-hbase5.apache.org,36017,1689175380873, RpcServer on jenkins-hbase5.apache.org/172.31.10.131:36017, sessionid=0x101c5bd87ea0002 2023-07-12 15:23:02,211 INFO [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(1637): Serving as jenkins-hbase5.apache.org,35567,1689175380829, RpcServer on jenkins-hbase5.apache.org/172.31.10.131:35567, sessionid=0x101c5bd87ea0001 2023-07-12 15:23:02,211 DEBUG [RS:0;jenkins-hbase5:35567] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 15:23:02,211 DEBUG [RS:1;jenkins-hbase5:36017] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 15:23:02,211 DEBUG [RS:0;jenkins-hbase5:35567] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:02,211 DEBUG [RS:1;jenkins-hbase5:36017] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:02,212 DEBUG [RS:0;jenkins-hbase5:35567] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,35567,1689175380829' 2023-07-12 15:23:02,212 DEBUG [RS:1;jenkins-hbase5:36017] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,36017,1689175380873' 2023-07-12 15:23:02,212 DEBUG [RS:1;jenkins-hbase5:36017] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-12 15:23:02,212 DEBUG [RS:0;jenkins-hbase5:35567] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-12 15:23:02,213 DEBUG [RS:1;jenkins-hbase5:36017] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-12 15:23:02,213 DEBUG [RS:0;jenkins-hbase5:35567] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-12 15:23:02,214 DEBUG [RS:0;jenkins-hbase5:35567] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 15:23:02,214 DEBUG [RS:1;jenkins-hbase5:36017] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 15:23:02,214 DEBUG [RS:0;jenkins-hbase5:35567] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 15:23:02,214 DEBUG [RS:1;jenkins-hbase5:36017] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 15:23:02,214 DEBUG [RS:0;jenkins-hbase5:35567] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:02,214 DEBUG [RS:1;jenkins-hbase5:36017] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:02,214 DEBUG [RS:0;jenkins-hbase5:35567] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,35567,1689175380829' 2023-07-12 15:23:02,214 DEBUG [RS:0;jenkins-hbase5:35567] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-12 15:23:02,214 DEBUG [RS:1;jenkins-hbase5:36017] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,36017,1689175380873' 2023-07-12 15:23:02,214 DEBUG [RS:1;jenkins-hbase5:36017] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-12 15:23:02,215 DEBUG [RS:0;jenkins-hbase5:35567] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-12 15:23:02,215 DEBUG [RS:1;jenkins-hbase5:36017] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-12 15:23:02,215 DEBUG [RS:0;jenkins-hbase5:35567] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 15:23:02,216 INFO [RS:0;jenkins-hbase5:35567] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 15:23:02,216 DEBUG [RS:1;jenkins-hbase5:36017] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 15:23:02,216 INFO [RS:0;jenkins-hbase5:35567] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 15:23:02,216 INFO [RS:1;jenkins-hbase5:36017] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 15:23:02,216 INFO [RS:1;jenkins-hbase5:36017] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 15:23:02,229 DEBUG [jenkins-hbase5:45903] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=2, allServersCount=2 2023-07-12 15:23:02,232 DEBUG [jenkins-hbase5:45903] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase5.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:23:02,238 DEBUG [jenkins-hbase5:45903] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:23:02,238 DEBUG [jenkins-hbase5:45903] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:23:02,238 DEBUG [jenkins-hbase5:45903] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:23:02,241 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase5.apache.org,35567,1689175380829, state=OPENING 2023-07-12 15:23:02,251 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-12 15:23:02,252 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:02,253 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-12 15:23:02,257 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase5.apache.org,35567,1689175380829}] 2023-07-12 15:23:02,327 INFO [RS:1;jenkins-hbase5:36017] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C36017%2C1689175380873, suffix=, logDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873, archiveDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs, maxLogs=10 2023-07-12 15:23:02,327 INFO [RS:0;jenkins-hbase5:35567] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C35567%2C1689175380829, suffix=, logDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,35567,1689175380829, archiveDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs, maxLogs=10 2023-07-12 15:23:02,352 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34197,DS-999b06e9-279a-47df-944d-eabe177df1c6,DISK] 2023-07-12 15:23:02,354 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38179,DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770,DISK] 2023-07-12 15:23:02,354 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38179,DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770,DISK] 2023-07-12 15:23:02,355 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34197,DS-999b06e9-279a-47df-944d-eabe177df1c6,DISK] 2023-07-12 15:23:02,364 INFO [RS:0;jenkins-hbase5:35567] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,35567,1689175380829/jenkins-hbase5.apache.org%2C35567%2C1689175380829.1689175382331 2023-07-12 15:23:02,364 DEBUG [RS:0;jenkins-hbase5:35567] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38179,DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770,DISK], DatanodeInfoWithStorage[127.0.0.1:34197,DS-999b06e9-279a-47df-944d-eabe177df1c6,DISK]] 2023-07-12 15:23:02,366 INFO [RS:1;jenkins-hbase5:36017] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873/jenkins-hbase5.apache.org%2C36017%2C1689175380873.1689175382331 2023-07-12 15:23:02,367 DEBUG [RS:1;jenkins-hbase5:36017] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34197,DS-999b06e9-279a-47df-944d-eabe177df1c6,DISK], DatanodeInfoWithStorage[127.0.0.1:38179,DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770,DISK]] 2023-07-12 15:23:02,443 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:02,445 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:23:02,449 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:58552, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 15:23:02,462 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-12 15:23:02,462 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:02,466 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C35567%2C1689175380829.meta, suffix=.meta, logDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,35567,1689175380829, archiveDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs, maxLogs=10 2023-07-12 15:23:02,483 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34197,DS-999b06e9-279a-47df-944d-eabe177df1c6,DISK] 2023-07-12 15:23:02,484 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38179,DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770,DISK] 2023-07-12 15:23:02,490 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,35567,1689175380829/jenkins-hbase5.apache.org%2C35567%2C1689175380829.meta.1689175382467.meta 2023-07-12 15:23:02,491 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34197,DS-999b06e9-279a-47df-944d-eabe177df1c6,DISK], DatanodeInfoWithStorage[127.0.0.1:38179,DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770,DISK]] 2023-07-12 15:23:02,492 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:02,494 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-12 15:23:02,510 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-12 15:23:02,551 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-12 15:23:02,556 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-12 15:23:02,556 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:02,556 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-12 15:23:02,556 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-12 15:23:02,560 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-12 15:23:02,562 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/info 2023-07-12 15:23:02,562 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/info 2023-07-12 15:23:02,562 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-12 15:23:02,564 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:02,564 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-12 15:23:02,565 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:23:02,565 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:23:02,566 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-12 15:23:02,567 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:02,567 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-12 15:23:02,568 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/table 2023-07-12 15:23:02,568 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/table 2023-07-12 15:23:02,569 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-12 15:23:02,569 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:02,571 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740 2023-07-12 15:23:02,574 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740 2023-07-12 15:23:02,577 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-12 15:23:02,580 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-12 15:23:02,581 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69229766, jitterRate=0.03160390257835388}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-12 15:23:02,581 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-12 15:23:02,594 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689175382435 2023-07-12 15:23:02,613 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-12 15:23:02,614 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-12 15:23:02,615 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase5.apache.org,35567,1689175380829, state=OPEN 2023-07-12 15:23:02,618 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-12 15:23:02,619 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-12 15:23:02,623 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-12 15:23:02,623 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase5.apache.org,35567,1689175380829 in 362 msec 2023-07-12 15:23:02,628 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-12 15:23:02,628 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 561 msec 2023-07-12 15:23:02,634 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 780 msec 2023-07-12 15:23:02,634 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689175382634, completionTime=-1 2023-07-12 15:23:02,634 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=2; waited=0ms, expected min=2 server(s), max=2 server(s), master is running 2023-07-12 15:23:02,635 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:23:02,695 DEBUG [hconnection-0x343b811a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:02,698 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:58566, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:02,717 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=2 2023-07-12 15:23:02,717 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689175442717 2023-07-12 15:23:02,717 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689175502717 2023-07-12 15:23:02,717 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 82 msec 2023-07-12 15:23:02,741 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,45903,1689175379564-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,741 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,45903,1689175379564-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,741 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,45903,1689175379564-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,743 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase5:45903, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,743 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:02,749 DEBUG [master/jenkins-hbase5:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-12 15:23:02,757 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-12 15:23:02,759 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-12 15:23:02,769 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-12 15:23:02,771 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:23:02,774 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:23:02,795 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:02,797 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34 empty. 2023-07-12 15:23:02,798 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:02,798 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-12 15:23:02,834 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-12 15:23:02,840 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2f5120c49287770b2e8c9c01ffb12e34, NAME => 'hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp 2023-07-12 15:23:02,855 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:02,855 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 2f5120c49287770b2e8c9c01ffb12e34, disabling compactions & flushes 2023-07-12 15:23:02,855 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:02,855 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:02,856 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. after waiting 0 ms 2023-07-12 15:23:02,856 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:02,856 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:02,856 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 2f5120c49287770b2e8c9c01ffb12e34: 2023-07-12 15:23:02,861 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:23:02,876 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689175382864"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689175382864"}]},"ts":"1689175382864"} 2023-07-12 15:23:02,910 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:23:02,912 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:23:02,917 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689175382912"}]},"ts":"1689175382912"} 2023-07-12 15:23:02,921 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-12 15:23:02,925 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase5.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:23:02,927 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:23:02,927 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:23:02,927 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:23:02,929 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=2f5120c49287770b2e8c9c01ffb12e34, ASSIGN}] 2023-07-12 15:23:02,932 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=2f5120c49287770b2e8c9c01ffb12e34, ASSIGN 2023-07-12 15:23:02,934 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=2f5120c49287770b2e8c9c01ffb12e34, ASSIGN; state=OFFLINE, location=jenkins-hbase5.apache.org,35567,1689175380829; forceNewPlan=false, retain=false 2023-07-12 15:23:03,086 INFO [jenkins-hbase5:45903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:23:03,087 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=2f5120c49287770b2e8c9c01ffb12e34, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:03,088 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689175383087"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175383087"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175383087"}]},"ts":"1689175383087"} 2023-07-12 15:23:03,092 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 2f5120c49287770b2e8c9c01ffb12e34, server=jenkins-hbase5.apache.org,35567,1689175380829}] 2023-07-12 15:23:03,252 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:03,253 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2f5120c49287770b2e8c9c01ffb12e34, NAME => 'hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:03,254 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:03,254 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:03,254 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:03,254 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:03,256 INFO [StoreOpener-2f5120c49287770b2e8c9c01ffb12e34-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:03,258 DEBUG [StoreOpener-2f5120c49287770b2e8c9c01ffb12e34-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34/info 2023-07-12 15:23:03,258 DEBUG [StoreOpener-2f5120c49287770b2e8c9c01ffb12e34-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34/info 2023-07-12 15:23:03,259 INFO [StoreOpener-2f5120c49287770b2e8c9c01ffb12e34-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2f5120c49287770b2e8c9c01ffb12e34 columnFamilyName info 2023-07-12 15:23:03,259 INFO [StoreOpener-2f5120c49287770b2e8c9c01ffb12e34-1] regionserver.HStore(310): Store=2f5120c49287770b2e8c9c01ffb12e34/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:03,261 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:03,262 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:03,266 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:03,269 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:23:03,270 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 2f5120c49287770b2e8c9c01ffb12e34; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65041934, jitterRate=-0.030799657106399536}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:23:03,270 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 2f5120c49287770b2e8c9c01ffb12e34: 2023-07-12 15:23:03,272 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34., pid=6, masterSystemTime=1689175383246 2023-07-12 15:23:03,277 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:03,277 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:03,279 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=2f5120c49287770b2e8c9c01ffb12e34, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:03,279 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689175383278"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175383278"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175383278"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175383278"}]},"ts":"1689175383278"} 2023-07-12 15:23:03,287 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-12 15:23:03,287 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 2f5120c49287770b2e8c9c01ffb12e34, server=jenkins-hbase5.apache.org,35567,1689175380829 in 191 msec 2023-07-12 15:23:03,290 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-12 15:23:03,291 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=2f5120c49287770b2e8c9c01ffb12e34, ASSIGN in 358 msec 2023-07-12 15:23:03,292 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:23:03,292 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689175383292"}]},"ts":"1689175383292"} 2023-07-12 15:23:03,295 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-12 15:23:03,300 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:23:03,302 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 540 msec 2023-07-12 15:23:03,372 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/namespace 2023-07-12 15:23:03,374 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/namespace 2023-07-12 15:23:03,374 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:03,426 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-12 15:23:03,444 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-12 15:23:03,452 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 40 msec 2023-07-12 15:23:03,460 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:23:03,478 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-12 15:23:03,485 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 23 msec 2023-07-12 15:23:03,497 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-12 15:23:03,500 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-12 15:23:03,501 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.573sec 2023-07-12 15:23:03,504 INFO [master/jenkins-hbase5:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-12 15:23:03,505 INFO [master/jenkins-hbase5:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-12 15:23:03,505 INFO [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-12 15:23:03,506 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,45903,1689175379564-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-12 15:23:03,507 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,45903,1689175379564-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-12 15:23:03,515 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-12 15:23:03,601 DEBUG [Listener at localhost/35683] zookeeper.ReadOnlyZKClient(139): Connect 0x5de352d4 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:03,606 DEBUG [Listener at localhost/35683] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1ea61f91, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:03,623 DEBUG [hconnection-0x4977ce4d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:03,638 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:58572, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:03,650 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase5.apache.org,45903,1689175379564 2023-07-12 15:23:03,650 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=4, rsPorts=, rsClass=null, numDataNodes=4, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-12 15:23:03,650 INFO [Listener at localhost/35683] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/cluster_28b984ac-22b7-2eea-77a9-387577b549fd, deleteOnExit=true 2023-07-12 15:23:03,651 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-12 15:23:03,663 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/test.cache.data in system properties and HBase conf 2023-07-12 15:23:03,663 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/hadoop.tmp.dir in system properties and HBase conf 2023-07-12 15:23:03,664 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/hadoop.log.dir in system properties and HBase conf 2023-07-12 15:23:03,664 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-12 15:23:03,664 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-12 15:23:03,664 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-12 15:23:03,664 DEBUG [Listener at localhost/35683] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-12 15:23:03,664 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-12 15:23:03,665 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-12 15:23:03,665 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-12 15:23:03,665 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-12 15:23:03,665 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-12 15:23:03,665 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-12 15:23:03,665 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-12 15:23:03,665 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-12 15:23:03,665 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-12 15:23:03,665 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/nfs.dump.dir in system properties and HBase conf 2023-07-12 15:23:03,666 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/java.io.tmpdir in system properties and HBase conf 2023-07-12 15:23:03,666 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-12 15:23:03,666 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-12 15:23:03,666 INFO [Listener at localhost/35683] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-12 15:23:03,672 WARN [Listener at localhost/35683] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-12 15:23:03,673 WARN [Listener at localhost/35683] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-12 15:23:03,718 WARN [Listener at localhost/35683] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 15:23:03,720 INFO [Listener at localhost/35683] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 15:23:03,726 INFO [Listener at localhost/35683] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/java.io.tmpdir/Jetty_localhost_43743_hdfs____odmh1d/webapp 2023-07-12 15:23:03,834 INFO [Listener at localhost/35683] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43743 2023-07-12 15:23:03,847 WARN [Listener at localhost/35683] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-12 15:23:03,847 WARN [Listener at localhost/35683] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-12 15:23:03,905 WARN [Listener at localhost/36223] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 15:23:03,924 WARN [Listener at localhost/36223] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-12 15:23:03,927 WARN [Listener at localhost/36223] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 15:23:03,929 INFO [Listener at localhost/36223] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 15:23:03,934 INFO [Listener at localhost/36223] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/java.io.tmpdir/Jetty_localhost_42401_datanode____k3jhi6/webapp 2023-07-12 15:23:04,031 INFO [Listener at localhost/36223] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42401 2023-07-12 15:23:04,041 WARN [Listener at localhost/38871] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 15:23:04,063 WARN [Listener at localhost/38871] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-12 15:23:04,066 WARN [Listener at localhost/38871] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 15:23:04,067 INFO [Listener at localhost/38871] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 15:23:04,072 INFO [Listener at localhost/38871] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/java.io.tmpdir/Jetty_localhost_36091_datanode____224ar0/webapp 2023-07-12 15:23:04,169 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x67a8a6704403f2f5: Processing first storage report for DS-f5196444-584a-4f4f-bc5b-b784f48827de from datanode 2a2a9a00-f087-4236-9d70-c0b5998a3c34 2023-07-12 15:23:04,170 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x67a8a6704403f2f5: from storage DS-f5196444-584a-4f4f-bc5b-b784f48827de node DatanodeRegistration(127.0.0.1:38115, datanodeUuid=2a2a9a00-f087-4236-9d70-c0b5998a3c34, infoPort=43389, infoSecurePort=0, ipcPort=38871, storageInfo=lv=-57;cid=testClusterID;nsid=57580031;c=1689175383676), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 15:23:04,170 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x67a8a6704403f2f5: Processing first storage report for DS-63f7f3b1-30b9-49a4-944a-f9dcd627f18f from datanode 2a2a9a00-f087-4236-9d70-c0b5998a3c34 2023-07-12 15:23:04,170 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x67a8a6704403f2f5: from storage DS-63f7f3b1-30b9-49a4-944a-f9dcd627f18f node DatanodeRegistration(127.0.0.1:38115, datanodeUuid=2a2a9a00-f087-4236-9d70-c0b5998a3c34, infoPort=43389, infoSecurePort=0, ipcPort=38871, storageInfo=lv=-57;cid=testClusterID;nsid=57580031;c=1689175383676), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 15:23:04,186 INFO [Listener at localhost/38871] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36091 2023-07-12 15:23:04,195 WARN [Listener at localhost/46477] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 15:23:04,269 WARN [Listener at localhost/46477] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-12 15:23:04,271 WARN [Listener at localhost/46477] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 15:23:04,272 INFO [Listener at localhost/46477] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 15:23:04,276 INFO [Listener at localhost/46477] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/java.io.tmpdir/Jetty_localhost_32847_datanode____yvy9cz/webapp 2023-07-12 15:23:04,366 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc74f662179a84eec: Processing first storage report for DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4 from datanode a7f7fb07-249b-4ff8-88f6-13227ee5a414 2023-07-12 15:23:04,366 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc74f662179a84eec: from storage DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4 node DatanodeRegistration(127.0.0.1:46687, datanodeUuid=a7f7fb07-249b-4ff8-88f6-13227ee5a414, infoPort=41473, infoSecurePort=0, ipcPort=46477, storageInfo=lv=-57;cid=testClusterID;nsid=57580031;c=1689175383676), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 15:23:04,367 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc74f662179a84eec: Processing first storage report for DS-7a6cf0b6-d2cd-4116-8a93-6fe613562df9 from datanode a7f7fb07-249b-4ff8-88f6-13227ee5a414 2023-07-12 15:23:04,367 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc74f662179a84eec: from storage DS-7a6cf0b6-d2cd-4116-8a93-6fe613562df9 node DatanodeRegistration(127.0.0.1:46687, datanodeUuid=a7f7fb07-249b-4ff8-88f6-13227ee5a414, infoPort=41473, infoSecurePort=0, ipcPort=46477, storageInfo=lv=-57;cid=testClusterID;nsid=57580031;c=1689175383676), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 15:23:04,385 INFO [Listener at localhost/46477] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:32847 2023-07-12 15:23:04,394 WARN [Listener at localhost/43267] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 15:23:04,412 WARN [Listener at localhost/43267] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-12 15:23:04,415 WARN [Listener at localhost/43267] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 15:23:04,416 INFO [Listener at localhost/43267] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 15:23:04,425 INFO [Listener at localhost/43267] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/java.io.tmpdir/Jetty_localhost_41667_datanode____.vgzen/webapp 2023-07-12 15:23:04,512 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb328a0b8b8211936: Processing first storage report for DS-222af625-1417-4c2e-a1f3-46a25780170d from datanode 6016f2dd-9b70-49ca-bffc-7a8f7192b3d2 2023-07-12 15:23:04,512 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb328a0b8b8211936: from storage DS-222af625-1417-4c2e-a1f3-46a25780170d node DatanodeRegistration(127.0.0.1:40607, datanodeUuid=6016f2dd-9b70-49ca-bffc-7a8f7192b3d2, infoPort=41459, infoSecurePort=0, ipcPort=43267, storageInfo=lv=-57;cid=testClusterID;nsid=57580031;c=1689175383676), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 15:23:04,512 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb328a0b8b8211936: Processing first storage report for DS-f1ef8d3e-5c4b-4364-8923-ac917f281e91 from datanode 6016f2dd-9b70-49ca-bffc-7a8f7192b3d2 2023-07-12 15:23:04,512 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb328a0b8b8211936: from storage DS-f1ef8d3e-5c4b-4364-8923-ac917f281e91 node DatanodeRegistration(127.0.0.1:40607, datanodeUuid=6016f2dd-9b70-49ca-bffc-7a8f7192b3d2, infoPort=41459, infoSecurePort=0, ipcPort=43267, storageInfo=lv=-57;cid=testClusterID;nsid=57580031;c=1689175383676), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 15:23:04,539 INFO [Listener at localhost/43267] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41667 2023-07-12 15:23:04,548 WARN [Listener at localhost/42735] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 15:23:04,654 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf65db9f846d56ed1: Processing first storage report for DS-70830774-8243-47cc-a229-fa9f5cfd830d from datanode b1a88679-2fde-4cbc-9840-014883a8d08e 2023-07-12 15:23:04,654 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf65db9f846d56ed1: from storage DS-70830774-8243-47cc-a229-fa9f5cfd830d node DatanodeRegistration(127.0.0.1:40757, datanodeUuid=b1a88679-2fde-4cbc-9840-014883a8d08e, infoPort=34233, infoSecurePort=0, ipcPort=42735, storageInfo=lv=-57;cid=testClusterID;nsid=57580031;c=1689175383676), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 15:23:04,655 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf65db9f846d56ed1: Processing first storage report for DS-aa7163cb-34c3-4634-a9ad-b14628c66742 from datanode b1a88679-2fde-4cbc-9840-014883a8d08e 2023-07-12 15:23:04,655 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf65db9f846d56ed1: from storage DS-aa7163cb-34c3-4634-a9ad-b14628c66742 node DatanodeRegistration(127.0.0.1:40757, datanodeUuid=b1a88679-2fde-4cbc-9840-014883a8d08e, infoPort=34233, infoSecurePort=0, ipcPort=42735, storageInfo=lv=-57;cid=testClusterID;nsid=57580031;c=1689175383676), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 15:23:04,658 DEBUG [Listener at localhost/42735] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed 2023-07-12 15:23:04,659 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:04,659 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:04,690 INFO [Listener at localhost/42735] util.FSUtils(471): Created version file at hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143 with version=8 2023-07-12 15:23:04,691 INFO [Listener at localhost/42735] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/hbase-staging 2023-07-12 15:23:04,693 INFO [Listener at localhost/42735] client.ConnectionUtils(127): master/jenkins-hbase5:0 server-side Connection retries=18 2023-07-12 15:23:04,693 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,693 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,693 INFO [Listener at localhost/42735] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:23:04,693 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,693 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:23:04,693 INFO [Listener at localhost/42735] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:23:04,695 INFO [Listener at localhost/42735] ipc.NettyRpcServer(120): Bind to /172.31.10.131:45779 2023-07-12 15:23:04,695 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:04,697 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:04,699 INFO [Listener at localhost/42735] zookeeper.RecoverableZooKeeper(93): Process identifier=master:45779 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:04,702 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:457790x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:04,702 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:45779-0x101c5bd87ea0007 connected 2023-07-12 15:23:04,714 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-12 15:23:04,715 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:04,715 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-12 15:23:04,715 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45779 2023-07-12 15:23:04,716 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45779 2023-07-12 15:23:04,716 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45779 2023-07-12 15:23:04,716 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45779 2023-07-12 15:23:04,716 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45779 2023-07-12 15:23:04,717 INFO [Listener at localhost/42735] master.HMaster(444): hbase.rootdir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143, hbase.cluster.distributed=false 2023-07-12 15:23:04,731 INFO [Listener at localhost/42735] client.ConnectionUtils(127): regionserver/jenkins-hbase5:0 server-side Connection retries=18 2023-07-12 15:23:04,731 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,732 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,732 INFO [Listener at localhost/42735] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:23:04,732 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,732 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:23:04,732 INFO [Listener at localhost/42735] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:23:04,733 INFO [Listener at localhost/42735] ipc.NettyRpcServer(120): Bind to /172.31.10.131:35897 2023-07-12 15:23:04,734 INFO [Listener at localhost/42735] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 15:23:04,735 DEBUG [Listener at localhost/42735] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 15:23:04,735 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:04,736 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:04,737 INFO [Listener at localhost/42735] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35897 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:04,740 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:358970x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:04,742 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:35897-0x101c5bd87ea0008 connected 2023-07-12 15:23:04,742 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-12 15:23:04,743 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:04,743 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-12 15:23:04,744 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35897 2023-07-12 15:23:04,744 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35897 2023-07-12 15:23:04,744 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35897 2023-07-12 15:23:04,744 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35897 2023-07-12 15:23:04,745 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35897 2023-07-12 15:23:04,756 INFO [Listener at localhost/42735] client.ConnectionUtils(127): regionserver/jenkins-hbase5:0 server-side Connection retries=18 2023-07-12 15:23:04,756 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,756 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,756 INFO [Listener at localhost/42735] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:23:04,757 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,757 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:23:04,757 INFO [Listener at localhost/42735] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:23:04,758 INFO [Listener at localhost/42735] ipc.NettyRpcServer(120): Bind to /172.31.10.131:42707 2023-07-12 15:23:04,759 INFO [Listener at localhost/42735] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 15:23:04,760 DEBUG [Listener at localhost/42735] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 15:23:04,761 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:04,762 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:04,763 INFO [Listener at localhost/42735] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42707 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:04,767 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:427070x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:04,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42707-0x101c5bd87ea0009 connected 2023-07-12 15:23:04,769 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-12 15:23:04,769 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:04,770 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-12 15:23:04,770 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42707 2023-07-12 15:23:04,771 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42707 2023-07-12 15:23:04,771 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42707 2023-07-12 15:23:04,771 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42707 2023-07-12 15:23:04,771 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42707 2023-07-12 15:23:04,783 INFO [Listener at localhost/42735] client.ConnectionUtils(127): regionserver/jenkins-hbase5:0 server-side Connection retries=18 2023-07-12 15:23:04,783 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,783 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,783 INFO [Listener at localhost/42735] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:23:04,784 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,784 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:23:04,784 INFO [Listener at localhost/42735] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:23:04,786 INFO [Listener at localhost/42735] ipc.NettyRpcServer(120): Bind to /172.31.10.131:38729 2023-07-12 15:23:04,786 INFO [Listener at localhost/42735] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 15:23:04,787 DEBUG [Listener at localhost/42735] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 15:23:04,788 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:04,789 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:04,790 INFO [Listener at localhost/42735] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38729 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:04,793 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:387290x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:04,794 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:387290x0, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-12 15:23:04,795 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38729-0x101c5bd87ea000a connected 2023-07-12 15:23:04,795 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:04,796 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-12 15:23:04,796 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38729 2023-07-12 15:23:04,797 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38729 2023-07-12 15:23:04,797 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38729 2023-07-12 15:23:04,798 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38729 2023-07-12 15:23:04,798 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38729 2023-07-12 15:23:04,809 INFO [Listener at localhost/42735] client.ConnectionUtils(127): regionserver/jenkins-hbase5:0 server-side Connection retries=18 2023-07-12 15:23:04,809 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,809 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,810 INFO [Listener at localhost/42735] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:23:04,810 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:04,810 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:23:04,810 INFO [Listener at localhost/42735] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:23:04,814 INFO [Listener at localhost/42735] ipc.NettyRpcServer(120): Bind to /172.31.10.131:33173 2023-07-12 15:23:04,815 INFO [Listener at localhost/42735] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 15:23:04,816 DEBUG [Listener at localhost/42735] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 15:23:04,816 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:04,817 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:04,818 INFO [Listener at localhost/42735] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33173 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:04,822 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:331730x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:04,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:33173-0x101c5bd87ea000b connected 2023-07-12 15:23:04,824 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-12 15:23:04,824 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:04,825 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-12 15:23:04,825 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33173 2023-07-12 15:23:04,825 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33173 2023-07-12 15:23:04,826 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33173 2023-07-12 15:23:04,826 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33173 2023-07-12 15:23:04,826 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33173 2023-07-12 15:23:04,828 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase5.apache.org,45779,1689175384692 2023-07-12 15:23:04,830 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-12 15:23:04,830 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase5.apache.org,45779,1689175384692 2023-07-12 15:23:04,832 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-12 15:23:04,832 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-12 15:23:04,832 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-12 15:23:04,832 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-12 15:23:04,832 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-12 15:23:04,834 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:04,834 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-12 15:23:04,837 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase5.apache.org,45779,1689175384692 from backup master directory 2023-07-12 15:23:04,837 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-12 15:23:04,839 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase5.apache.org,45779,1689175384692 2023-07-12 15:23:04,839 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-12 15:23:04,839 WARN [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:23:04,839 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase5.apache.org,45779,1689175384692 2023-07-12 15:23:04,870 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/hbase.id with ID: 5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:23:04,888 INFO [master/jenkins-hbase5:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:04,894 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:04,914 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x51755a69 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:04,918 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@d24412d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:04,919 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-12 15:23:04,919 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-12 15:23:04,919 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:04,921 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store-tmp 2023-07-12 15:23:04,945 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:04,945 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-12 15:23:04,945 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:04,945 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:04,945 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-12 15:23:04,945 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:04,945 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:04,945 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:23:04,946 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,45779,1689175384692 2023-07-12 15:23:04,950 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C45779%2C1689175384692, suffix=, logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,45779,1689175384692, archiveDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/oldWALs, maxLogs=10 2023-07-12 15:23:04,969 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46687,DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4,DISK] 2023-07-12 15:23:04,971 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK] 2023-07-12 15:23:04,972 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK] 2023-07-12 15:23:04,976 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,45779,1689175384692/jenkins-hbase5.apache.org%2C45779%2C1689175384692.1689175384950 2023-07-12 15:23:04,976 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46687,DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4,DISK], DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK], DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK]] 2023-07-12 15:23:04,976 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:04,976 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:04,976 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:04,976 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:04,979 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:04,981 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-12 15:23:04,982 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-12 15:23:04,983 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:04,984 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:04,985 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:04,989 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:04,992 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:23:04,993 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67588539, jitterRate=0.0071477144956588745}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:23:04,993 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:23:04,996 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-12 15:23:04,998 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-12 15:23:04,998 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-12 15:23:04,998 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:23:04,999 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:23:05,000 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-07-12 15:23:05,000 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-12 15:23:05,001 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-12 15:23:05,003 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-12 15:23:05,013 INFO [master/jenkins-hbase5:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-12 15:23:05,013 INFO [master/jenkins-hbase5:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-12 15:23:05,014 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-12 15:23:05,014 INFO [master/jenkins-hbase5:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-12 15:23:05,014 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-12 15:23:05,017 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:05,018 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-12 15:23:05,018 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-12 15:23:05,019 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-12 15:23:05,021 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-12 15:23:05,021 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-12 15:23:05,021 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-12 15:23:05,021 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-12 15:23:05,021 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-12 15:23:05,021 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:05,022 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase5.apache.org,45779,1689175384692, sessionid=0x101c5bd87ea0007, setting cluster-up flag (Was=false) 2023-07-12 15:23:05,027 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:05,037 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-12 15:23:05,038 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase5.apache.org,45779,1689175384692 2023-07-12 15:23:05,041 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:05,048 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-12 15:23:05,048 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase5.apache.org,45779,1689175384692 2023-07-12 15:23:05,049 WARN [master/jenkins-hbase5:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.hbase-snapshot/.tmp 2023-07-12 15:23:05,052 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-12 15:23:05,052 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:05,052 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:05,053 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:05,053 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:05,053 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase5:0, corePoolSize=10, maxPoolSize=10 2023-07-12 15:23:05,053 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,053 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:23:05,053 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,057 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689175415057 2023-07-12 15:23:05,057 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-12 15:23:05,057 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-12 15:23:05,057 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-12 15:23:05,057 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-12 15:23:05,057 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-12 15:23:05,057 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-12 15:23:05,057 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,058 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-12 15:23:05,058 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-12 15:23:05,058 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-12 15:23:05,059 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-12 15:23:05,059 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-12 15:23:05,059 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-12 15:23:05,059 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-12 15:23:05,059 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175385059,5,FailOnTimeoutGroup] 2023-07-12 15:23:05,059 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175385059,5,FailOnTimeoutGroup] 2023-07-12 15:23:05,059 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,059 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-12 15:23:05,059 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,059 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,060 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-12 15:23:05,085 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-12 15:23:05,086 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-12 15:23:05,086 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143 2023-07-12 15:23:05,103 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:05,105 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-12 15:23:05,107 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info 2023-07-12 15:23:05,108 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-12 15:23:05,109 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:05,109 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-12 15:23:05,110 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:23:05,111 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-12 15:23:05,111 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:05,111 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-12 15:23:05,113 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/table 2023-07-12 15:23:05,113 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-12 15:23:05,114 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:05,116 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740 2023-07-12 15:23:05,116 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740 2023-07-12 15:23:05,119 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-12 15:23:05,121 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-12 15:23:05,123 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:23:05,124 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69336277, jitterRate=0.03319104015827179}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-12 15:23:05,124 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-12 15:23:05,124 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-12 15:23:05,124 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-12 15:23:05,124 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-12 15:23:05,124 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-12 15:23:05,124 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-12 15:23:05,125 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-12 15:23:05,125 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-12 15:23:05,127 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-12 15:23:05,127 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-12 15:23:05,128 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-12 15:23:05,129 INFO [RS:0;jenkins-hbase5:35897] regionserver.HRegionServer(951): ClusterId : 5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:23:05,129 INFO [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(951): ClusterId : 5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:23:05,130 DEBUG [RS:0;jenkins-hbase5:35897] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 15:23:05,129 INFO [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(951): ClusterId : 5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:23:05,129 INFO [RS:2;jenkins-hbase5:38729] regionserver.HRegionServer(951): ClusterId : 5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:23:05,132 DEBUG [RS:3;jenkins-hbase5:33173] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 15:23:05,131 DEBUG [RS:1;jenkins-hbase5:42707] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 15:23:05,133 DEBUG [RS:2;jenkins-hbase5:38729] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 15:23:05,134 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-12 15:23:05,137 DEBUG [RS:0;jenkins-hbase5:35897] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 15:23:05,137 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-12 15:23:05,137 DEBUG [RS:2;jenkins-hbase5:38729] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 15:23:05,137 DEBUG [RS:0;jenkins-hbase5:35897] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 15:23:05,137 DEBUG [RS:2;jenkins-hbase5:38729] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 15:23:05,138 DEBUG [RS:3;jenkins-hbase5:33173] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 15:23:05,138 DEBUG [RS:3;jenkins-hbase5:33173] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 15:23:05,138 DEBUG [RS:1;jenkins-hbase5:42707] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 15:23:05,138 DEBUG [RS:1;jenkins-hbase5:42707] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 15:23:05,143 DEBUG [RS:0;jenkins-hbase5:35897] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 15:23:05,144 DEBUG [RS:2;jenkins-hbase5:38729] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 15:23:05,144 DEBUG [RS:3;jenkins-hbase5:33173] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 15:23:05,145 DEBUG [RS:0;jenkins-hbase5:35897] zookeeper.ReadOnlyZKClient(139): Connect 0x24128b82 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:05,146 DEBUG [RS:1;jenkins-hbase5:42707] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 15:23:05,147 DEBUG [RS:2;jenkins-hbase5:38729] zookeeper.ReadOnlyZKClient(139): Connect 0x7d9c25a0 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:05,150 DEBUG [RS:3;jenkins-hbase5:33173] zookeeper.ReadOnlyZKClient(139): Connect 0x21119e79 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:05,150 DEBUG [RS:1;jenkins-hbase5:42707] zookeeper.ReadOnlyZKClient(139): Connect 0x7bf78aee to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:05,158 DEBUG [RS:0;jenkins-hbase5:35897] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2ad57170, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:05,158 DEBUG [RS:0;jenkins-hbase5:35897] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2eeae0f2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase5.apache.org/172.31.10.131:0 2023-07-12 15:23:05,161 DEBUG [RS:2;jenkins-hbase5:38729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@735bc463, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:05,161 DEBUG [RS:1;jenkins-hbase5:42707] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3aca80d5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:05,161 DEBUG [RS:3;jenkins-hbase5:33173] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@602c9442, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:05,162 DEBUG [RS:1;jenkins-hbase5:42707] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4cddcc0e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase5.apache.org/172.31.10.131:0 2023-07-12 15:23:05,161 DEBUG [RS:2;jenkins-hbase5:38729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@77644ed0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase5.apache.org/172.31.10.131:0 2023-07-12 15:23:05,162 DEBUG [RS:3;jenkins-hbase5:33173] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7c498c49, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase5.apache.org/172.31.10.131:0 2023-07-12 15:23:05,170 DEBUG [RS:0;jenkins-hbase5:35897] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase5:35897 2023-07-12 15:23:05,170 INFO [RS:0;jenkins-hbase5:35897] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 15:23:05,170 INFO [RS:0;jenkins-hbase5:35897] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 15:23:05,170 DEBUG [RS:0;jenkins-hbase5:35897] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 15:23:05,171 DEBUG [RS:1;jenkins-hbase5:42707] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase5:42707 2023-07-12 15:23:05,171 INFO [RS:1;jenkins-hbase5:42707] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 15:23:05,171 INFO [RS:1;jenkins-hbase5:42707] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 15:23:05,171 DEBUG [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 15:23:05,171 INFO [RS:0;jenkins-hbase5:35897] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase5.apache.org,45779,1689175384692 with isa=jenkins-hbase5.apache.org/172.31.10.131:35897, startcode=1689175384731 2023-07-12 15:23:05,172 DEBUG [RS:0;jenkins-hbase5:35897] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 15:23:05,172 INFO [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase5.apache.org,45779,1689175384692 with isa=jenkins-hbase5.apache.org/172.31.10.131:42707, startcode=1689175384756 2023-07-12 15:23:05,172 DEBUG [RS:1;jenkins-hbase5:42707] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 15:23:05,173 DEBUG [RS:2;jenkins-hbase5:38729] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase5:38729 2023-07-12 15:23:05,173 INFO [RS:2;jenkins-hbase5:38729] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 15:23:05,173 INFO [RS:2;jenkins-hbase5:38729] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 15:23:05,173 DEBUG [RS:2;jenkins-hbase5:38729] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 15:23:05,175 INFO [RS:2;jenkins-hbase5:38729] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase5.apache.org,45779,1689175384692 with isa=jenkins-hbase5.apache.org/172.31.10.131:38729, startcode=1689175384783 2023-07-12 15:23:05,175 DEBUG [RS:2;jenkins-hbase5:38729] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 15:23:05,175 DEBUG [RS:3;jenkins-hbase5:33173] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase5:33173 2023-07-12 15:23:05,175 INFO [RS:3;jenkins-hbase5:33173] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 15:23:05,176 INFO [RS:3;jenkins-hbase5:33173] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 15:23:05,176 DEBUG [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 15:23:05,177 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:45661, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 15:23:05,178 INFO [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase5.apache.org,45779,1689175384692 with isa=jenkins-hbase5.apache.org/172.31.10.131:33173, startcode=1689175384809 2023-07-12 15:23:05,178 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:58511, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 15:23:05,178 DEBUG [RS:3;jenkins-hbase5:33173] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 15:23:05,179 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45779] master.ServerManager(394): Registering regionserver=jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:05,179 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:36785, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 15:23:05,180 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45779] master.ServerManager(394): Registering regionserver=jenkins-hbase5.apache.org,35897,1689175384731 2023-07-12 15:23:05,180 DEBUG [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143 2023-07-12 15:23:05,180 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45779] master.ServerManager(394): Registering regionserver=jenkins-hbase5.apache.org,38729,1689175384783 2023-07-12 15:23:05,180 DEBUG [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:36223 2023-07-12 15:23:05,180 DEBUG [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 15:23:05,180 DEBUG [RS:0;jenkins-hbase5:35897] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143 2023-07-12 15:23:05,180 DEBUG [RS:2;jenkins-hbase5:38729] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143 2023-07-12 15:23:05,181 DEBUG [RS:0;jenkins-hbase5:35897] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:36223 2023-07-12 15:23:05,181 DEBUG [RS:2;jenkins-hbase5:38729] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:36223 2023-07-12 15:23:05,181 DEBUG [RS:0;jenkins-hbase5:35897] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 15:23:05,181 DEBUG [RS:2;jenkins-hbase5:38729] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 15:23:05,183 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:34559, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 15:23:05,183 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:05,183 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45779] master.ServerManager(394): Registering regionserver=jenkins-hbase5.apache.org,33173,1689175384809 2023-07-12 15:23:05,184 DEBUG [RS:1;jenkins-hbase5:42707] zookeeper.ZKUtil(162): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:05,184 DEBUG [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143 2023-07-12 15:23:05,185 DEBUG [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:36223 2023-07-12 15:23:05,185 DEBUG [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 15:23:05,184 WARN [RS:1;jenkins-hbase5:42707] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:23:05,185 INFO [RS:1;jenkins-hbase5:42707] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:05,185 DEBUG [RS:0;jenkins-hbase5:35897] zookeeper.ZKUtil(162): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,35897,1689175384731 2023-07-12 15:23:05,185 WARN [RS:0;jenkins-hbase5:35897] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:23:05,185 INFO [RS:0;jenkins-hbase5:35897] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:05,185 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase5.apache.org,38729,1689175384783] 2023-07-12 15:23:05,185 DEBUG [RS:0;jenkins-hbase5:35897] regionserver.HRegionServer(1948): logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,35897,1689175384731 2023-07-12 15:23:05,185 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase5.apache.org,42707,1689175384756] 2023-07-12 15:23:05,185 DEBUG [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(1948): logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:05,185 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase5.apache.org,35897,1689175384731] 2023-07-12 15:23:05,186 DEBUG [RS:2;jenkins-hbase5:38729] zookeeper.ZKUtil(162): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,38729,1689175384783 2023-07-12 15:23:05,186 WARN [RS:2;jenkins-hbase5:38729] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:23:05,186 INFO [RS:2;jenkins-hbase5:38729] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:05,186 DEBUG [RS:2;jenkins-hbase5:38729] regionserver.HRegionServer(1948): logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,38729,1689175384783 2023-07-12 15:23:05,187 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:05,187 DEBUG [RS:3;jenkins-hbase5:33173] zookeeper.ZKUtil(162): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,33173,1689175384809 2023-07-12 15:23:05,188 WARN [RS:3;jenkins-hbase5:33173] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:23:05,188 INFO [RS:3;jenkins-hbase5:33173] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:05,188 DEBUG [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(1948): logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33173,1689175384809 2023-07-12 15:23:05,194 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase5.apache.org,33173,1689175384809] 2023-07-12 15:23:05,205 DEBUG [RS:0;jenkins-hbase5:35897] zookeeper.ZKUtil(162): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,33173,1689175384809 2023-07-12 15:23:05,205 DEBUG [RS:3;jenkins-hbase5:33173] zookeeper.ZKUtil(162): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,33173,1689175384809 2023-07-12 15:23:05,205 DEBUG [RS:1;jenkins-hbase5:42707] zookeeper.ZKUtil(162): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,33173,1689175384809 2023-07-12 15:23:05,205 DEBUG [RS:0;jenkins-hbase5:35897] zookeeper.ZKUtil(162): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,35897,1689175384731 2023-07-12 15:23:05,206 DEBUG [RS:0;jenkins-hbase5:35897] zookeeper.ZKUtil(162): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,38729,1689175384783 2023-07-12 15:23:05,206 DEBUG [RS:0;jenkins-hbase5:35897] zookeeper.ZKUtil(162): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:05,207 DEBUG [RS:1;jenkins-hbase5:42707] zookeeper.ZKUtil(162): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,35897,1689175384731 2023-07-12 15:23:05,207 DEBUG [RS:0;jenkins-hbase5:35897] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-12 15:23:05,208 INFO [RS:0;jenkins-hbase5:35897] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 15:23:05,208 DEBUG [RS:3;jenkins-hbase5:33173] zookeeper.ZKUtil(162): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,35897,1689175384731 2023-07-12 15:23:05,208 DEBUG [RS:1;jenkins-hbase5:42707] zookeeper.ZKUtil(162): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,38729,1689175384783 2023-07-12 15:23:05,208 DEBUG [RS:3;jenkins-hbase5:33173] zookeeper.ZKUtil(162): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,38729,1689175384783 2023-07-12 15:23:05,209 DEBUG [RS:3;jenkins-hbase5:33173] zookeeper.ZKUtil(162): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:05,209 DEBUG [RS:1;jenkins-hbase5:42707] zookeeper.ZKUtil(162): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:05,210 DEBUG [RS:2;jenkins-hbase5:38729] zookeeper.ZKUtil(162): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,33173,1689175384809 2023-07-12 15:23:05,210 DEBUG [RS:3;jenkins-hbase5:33173] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-12 15:23:05,211 DEBUG [RS:1;jenkins-hbase5:42707] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-12 15:23:05,211 INFO [RS:3;jenkins-hbase5:33173] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 15:23:05,211 INFO [RS:1;jenkins-hbase5:42707] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 15:23:05,211 DEBUG [RS:2;jenkins-hbase5:38729] zookeeper.ZKUtil(162): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,35897,1689175384731 2023-07-12 15:23:05,211 DEBUG [RS:2;jenkins-hbase5:38729] zookeeper.ZKUtil(162): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,38729,1689175384783 2023-07-12 15:23:05,212 DEBUG [RS:2;jenkins-hbase5:38729] zookeeper.ZKUtil(162): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:05,215 DEBUG [RS:2;jenkins-hbase5:38729] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-12 15:23:05,215 INFO [RS:2;jenkins-hbase5:38729] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 15:23:05,217 INFO [RS:0;jenkins-hbase5:35897] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 15:23:05,219 INFO [RS:2;jenkins-hbase5:38729] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 15:23:05,219 INFO [RS:3;jenkins-hbase5:33173] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 15:23:05,219 INFO [RS:1;jenkins-hbase5:42707] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 15:23:05,219 INFO [RS:1;jenkins-hbase5:42707] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 15:23:05,219 INFO [RS:2;jenkins-hbase5:38729] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 15:23:05,219 INFO [RS:1;jenkins-hbase5:42707] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,219 INFO [RS:2;jenkins-hbase5:38729] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,223 INFO [RS:2;jenkins-hbase5:38729] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 15:23:05,223 INFO [RS:0;jenkins-hbase5:35897] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 15:23:05,227 INFO [RS:3;jenkins-hbase5:33173] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 15:23:05,227 INFO [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 15:23:05,228 INFO [RS:3;jenkins-hbase5:33173] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,228 INFO [RS:0;jenkins-hbase5:35897] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,231 INFO [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 15:23:05,231 INFO [RS:1;jenkins-hbase5:42707] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,231 INFO [RS:2;jenkins-hbase5:38729] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,232 INFO [RS:0;jenkins-hbase5:35897] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 15:23:05,233 DEBUG [RS:1;jenkins-hbase5:42707] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,233 DEBUG [RS:2;jenkins-hbase5:38729] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,233 DEBUG [RS:1;jenkins-hbase5:42707] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,233 DEBUG [RS:1;jenkins-hbase5:42707] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,233 DEBUG [RS:2;jenkins-hbase5:38729] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,233 DEBUG [RS:1;jenkins-hbase5:42707] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,234 DEBUG [RS:2;jenkins-hbase5:38729] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,234 DEBUG [RS:1;jenkins-hbase5:42707] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,234 DEBUG [RS:2;jenkins-hbase5:38729] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,234 DEBUG [RS:1;jenkins-hbase5:42707] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase5:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:23:05,234 DEBUG [RS:2;jenkins-hbase5:38729] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,234 DEBUG [RS:1;jenkins-hbase5:42707] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,234 DEBUG [RS:2;jenkins-hbase5:38729] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase5:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:23:05,234 DEBUG [RS:1;jenkins-hbase5:42707] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,234 DEBUG [RS:2;jenkins-hbase5:38729] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,235 DEBUG [RS:1;jenkins-hbase5:42707] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,235 DEBUG [RS:2;jenkins-hbase5:38729] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,235 DEBUG [RS:1;jenkins-hbase5:42707] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,235 DEBUG [RS:2;jenkins-hbase5:38729] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,234 INFO [RS:0;jenkins-hbase5:35897] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,234 INFO [RS:3;jenkins-hbase5:33173] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,235 DEBUG [RS:2;jenkins-hbase5:38729] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,243 DEBUG [RS:3;jenkins-hbase5:33173] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,243 DEBUG [RS:0;jenkins-hbase5:35897] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,244 DEBUG [RS:3;jenkins-hbase5:33173] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,245 INFO [RS:2;jenkins-hbase5:38729] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,245 DEBUG [RS:3;jenkins-hbase5:33173] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,245 INFO [RS:2;jenkins-hbase5:38729] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,245 DEBUG [RS:3;jenkins-hbase5:33173] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,244 INFO [RS:1;jenkins-hbase5:42707] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,245 DEBUG [RS:3;jenkins-hbase5:33173] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,245 INFO [RS:1;jenkins-hbase5:42707] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,245 INFO [RS:2;jenkins-hbase5:38729] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,245 DEBUG [RS:0;jenkins-hbase5:35897] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,245 INFO [RS:1;jenkins-hbase5:42707] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,245 DEBUG [RS:3;jenkins-hbase5:33173] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase5:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:23:05,245 DEBUG [RS:0;jenkins-hbase5:35897] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,246 DEBUG [RS:3;jenkins-hbase5:33173] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,246 DEBUG [RS:0;jenkins-hbase5:35897] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,246 DEBUG [RS:3;jenkins-hbase5:33173] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,246 DEBUG [RS:0;jenkins-hbase5:35897] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,246 DEBUG [RS:3;jenkins-hbase5:33173] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,246 DEBUG [RS:0;jenkins-hbase5:35897] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase5:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:23:05,246 DEBUG [RS:3;jenkins-hbase5:33173] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,246 DEBUG [RS:0;jenkins-hbase5:35897] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,246 DEBUG [RS:0;jenkins-hbase5:35897] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,246 DEBUG [RS:0;jenkins-hbase5:35897] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,246 DEBUG [RS:0;jenkins-hbase5:35897] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:05,248 INFO [RS:3;jenkins-hbase5:33173] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,248 INFO [RS:3;jenkins-hbase5:33173] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,248 INFO [RS:3;jenkins-hbase5:33173] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,250 INFO [RS:0;jenkins-hbase5:35897] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,250 INFO [RS:0;jenkins-hbase5:35897] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,250 INFO [RS:0;jenkins-hbase5:35897] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,262 INFO [RS:1;jenkins-hbase5:42707] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 15:23:05,263 INFO [RS:1;jenkins-hbase5:42707] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,42707,1689175384756-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,263 INFO [RS:3;jenkins-hbase5:33173] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 15:23:05,263 INFO [RS:2;jenkins-hbase5:38729] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 15:23:05,263 INFO [RS:3;jenkins-hbase5:33173] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,33173,1689175384809-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,263 INFO [RS:2;jenkins-hbase5:38729] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,38729,1689175384783-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,271 INFO [RS:0;jenkins-hbase5:35897] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 15:23:05,271 INFO [RS:0;jenkins-hbase5:35897] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,35897,1689175384731-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,277 INFO [RS:3;jenkins-hbase5:33173] regionserver.Replication(203): jenkins-hbase5.apache.org,33173,1689175384809 started 2023-07-12 15:23:05,277 INFO [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(1637): Serving as jenkins-hbase5.apache.org,33173,1689175384809, RpcServer on jenkins-hbase5.apache.org/172.31.10.131:33173, sessionid=0x101c5bd87ea000b 2023-07-12 15:23:05,277 DEBUG [RS:3;jenkins-hbase5:33173] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 15:23:05,277 DEBUG [RS:3;jenkins-hbase5:33173] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase5.apache.org,33173,1689175384809 2023-07-12 15:23:05,277 DEBUG [RS:3;jenkins-hbase5:33173] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,33173,1689175384809' 2023-07-12 15:23:05,277 DEBUG [RS:3;jenkins-hbase5:33173] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-12 15:23:05,278 DEBUG [RS:3;jenkins-hbase5:33173] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-12 15:23:05,280 INFO [RS:2;jenkins-hbase5:38729] regionserver.Replication(203): jenkins-hbase5.apache.org,38729,1689175384783 started 2023-07-12 15:23:05,280 INFO [RS:2;jenkins-hbase5:38729] regionserver.HRegionServer(1637): Serving as jenkins-hbase5.apache.org,38729,1689175384783, RpcServer on jenkins-hbase5.apache.org/172.31.10.131:38729, sessionid=0x101c5bd87ea000a 2023-07-12 15:23:05,280 DEBUG [RS:2;jenkins-hbase5:38729] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 15:23:05,280 DEBUG [RS:2;jenkins-hbase5:38729] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase5.apache.org,38729,1689175384783 2023-07-12 15:23:05,280 DEBUG [RS:3;jenkins-hbase5:33173] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 15:23:05,280 DEBUG [RS:3;jenkins-hbase5:33173] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 15:23:05,280 DEBUG [RS:2;jenkins-hbase5:38729] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,38729,1689175384783' 2023-07-12 15:23:05,280 DEBUG [RS:2;jenkins-hbase5:38729] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-12 15:23:05,280 DEBUG [RS:3;jenkins-hbase5:33173] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase5.apache.org,33173,1689175384809 2023-07-12 15:23:05,280 DEBUG [RS:3;jenkins-hbase5:33173] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,33173,1689175384809' 2023-07-12 15:23:05,281 DEBUG [RS:3;jenkins-hbase5:33173] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-12 15:23:05,281 DEBUG [RS:2;jenkins-hbase5:38729] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-12 15:23:05,281 INFO [RS:1;jenkins-hbase5:42707] regionserver.Replication(203): jenkins-hbase5.apache.org,42707,1689175384756 started 2023-07-12 15:23:05,281 DEBUG [RS:3;jenkins-hbase5:33173] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-12 15:23:05,281 INFO [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(1637): Serving as jenkins-hbase5.apache.org,42707,1689175384756, RpcServer on jenkins-hbase5.apache.org/172.31.10.131:42707, sessionid=0x101c5bd87ea0009 2023-07-12 15:23:05,281 DEBUG [RS:1;jenkins-hbase5:42707] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 15:23:05,281 DEBUG [RS:1;jenkins-hbase5:42707] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:05,281 DEBUG [RS:1;jenkins-hbase5:42707] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,42707,1689175384756' 2023-07-12 15:23:05,281 DEBUG [RS:1;jenkins-hbase5:42707] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-12 15:23:05,282 DEBUG [RS:3;jenkins-hbase5:33173] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 15:23:05,282 INFO [RS:3;jenkins-hbase5:33173] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 15:23:05,282 DEBUG [RS:2;jenkins-hbase5:38729] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 15:23:05,282 DEBUG [RS:2;jenkins-hbase5:38729] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 15:23:05,282 DEBUG [RS:2;jenkins-hbase5:38729] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase5.apache.org,38729,1689175384783 2023-07-12 15:23:05,282 INFO [RS:3;jenkins-hbase5:33173] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 15:23:05,282 DEBUG [RS:1;jenkins-hbase5:42707] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-12 15:23:05,282 DEBUG [RS:2;jenkins-hbase5:38729] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,38729,1689175384783' 2023-07-12 15:23:05,282 DEBUG [RS:2;jenkins-hbase5:38729] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-12 15:23:05,282 DEBUG [RS:1;jenkins-hbase5:42707] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 15:23:05,282 DEBUG [RS:1;jenkins-hbase5:42707] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 15:23:05,282 DEBUG [RS:1;jenkins-hbase5:42707] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:05,282 DEBUG [RS:1;jenkins-hbase5:42707] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,42707,1689175384756' 2023-07-12 15:23:05,283 DEBUG [RS:1;jenkins-hbase5:42707] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-12 15:23:05,283 DEBUG [RS:2;jenkins-hbase5:38729] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-12 15:23:05,283 DEBUG [RS:1;jenkins-hbase5:42707] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-12 15:23:05,283 DEBUG [RS:2;jenkins-hbase5:38729] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 15:23:05,283 INFO [RS:2;jenkins-hbase5:38729] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 15:23:05,283 INFO [RS:2;jenkins-hbase5:38729] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 15:23:05,284 DEBUG [RS:1;jenkins-hbase5:42707] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 15:23:05,284 INFO [RS:1;jenkins-hbase5:42707] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 15:23:05,284 INFO [RS:1;jenkins-hbase5:42707] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 15:23:05,287 DEBUG [jenkins-hbase5:45779] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=4, allServersCount=4 2023-07-12 15:23:05,288 DEBUG [jenkins-hbase5:45779] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase5.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:23:05,290 DEBUG [jenkins-hbase5:45779] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:23:05,290 DEBUG [jenkins-hbase5:45779] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:23:05,290 DEBUG [jenkins-hbase5:45779] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:23:05,290 DEBUG [jenkins-hbase5:45779] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-12 15:23:05,290 DEBUG [jenkins-hbase5:45779] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:23:05,291 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase5.apache.org,33173,1689175384809, state=OPENING 2023-07-12 15:23:05,292 INFO [RS:0;jenkins-hbase5:35897] regionserver.Replication(203): jenkins-hbase5.apache.org,35897,1689175384731 started 2023-07-12 15:23:05,292 INFO [RS:0;jenkins-hbase5:35897] regionserver.HRegionServer(1637): Serving as jenkins-hbase5.apache.org,35897,1689175384731, RpcServer on jenkins-hbase5.apache.org/172.31.10.131:35897, sessionid=0x101c5bd87ea0008 2023-07-12 15:23:05,292 DEBUG [RS:0;jenkins-hbase5:35897] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 15:23:05,292 DEBUG [RS:0;jenkins-hbase5:35897] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase5.apache.org,35897,1689175384731 2023-07-12 15:23:05,292 DEBUG [RS:0;jenkins-hbase5:35897] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,35897,1689175384731' 2023-07-12 15:23:05,292 DEBUG [RS:0;jenkins-hbase5:35897] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-12 15:23:05,293 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-12 15:23:05,293 DEBUG [RS:0;jenkins-hbase5:35897] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-12 15:23:05,294 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:05,294 DEBUG [RS:0;jenkins-hbase5:35897] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 15:23:05,294 DEBUG [RS:0;jenkins-hbase5:35897] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 15:23:05,294 DEBUG [RS:0;jenkins-hbase5:35897] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase5.apache.org,35897,1689175384731 2023-07-12 15:23:05,294 DEBUG [RS:0;jenkins-hbase5:35897] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,35897,1689175384731' 2023-07-12 15:23:05,294 DEBUG [RS:0;jenkins-hbase5:35897] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-12 15:23:05,295 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-12 15:23:05,295 DEBUG [RS:0;jenkins-hbase5:35897] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-12 15:23:05,295 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase5.apache.org,33173,1689175384809}] 2023-07-12 15:23:05,295 DEBUG [RS:0;jenkins-hbase5:35897] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 15:23:05,295 INFO [RS:0;jenkins-hbase5:35897] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 15:23:05,296 INFO [RS:0;jenkins-hbase5:35897] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 15:23:05,385 INFO [RS:3;jenkins-hbase5:33173] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C33173%2C1689175384809, suffix=, logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33173,1689175384809, archiveDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs, maxLogs=10 2023-07-12 15:23:05,386 INFO [RS:2;jenkins-hbase5:38729] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C38729%2C1689175384783, suffix=, logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,38729,1689175384783, archiveDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs, maxLogs=10 2023-07-12 15:23:05,386 INFO [RS:1;jenkins-hbase5:42707] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C42707%2C1689175384756, suffix=, logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,42707,1689175384756, archiveDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs, maxLogs=10 2023-07-12 15:23:05,401 INFO [RS:0;jenkins-hbase5:35897] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C35897%2C1689175384731, suffix=, logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,35897,1689175384731, archiveDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs, maxLogs=10 2023-07-12 15:23:05,415 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK] 2023-07-12 15:23:05,415 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK] 2023-07-12 15:23:05,415 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46687,DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4,DISK] 2023-07-12 15:23:05,421 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK] 2023-07-12 15:23:05,428 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK] 2023-07-12 15:23:05,429 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK] 2023-07-12 15:23:05,429 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK] 2023-07-12 15:23:05,429 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46687,DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4,DISK] 2023-07-12 15:23:05,429 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK] 2023-07-12 15:23:05,443 INFO [RS:3;jenkins-hbase5:33173] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33173,1689175384809/jenkins-hbase5.apache.org%2C33173%2C1689175384809.1689175385386 2023-07-12 15:23:05,443 INFO [RS:2;jenkins-hbase5:38729] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,38729,1689175384783/jenkins-hbase5.apache.org%2C38729%2C1689175384783.1689175385388 2023-07-12 15:23:05,443 DEBUG [RS:3;jenkins-hbase5:33173] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK], DatanodeInfoWithStorage[127.0.0.1:46687,DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4,DISK], DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK]] 2023-07-12 15:23:05,445 DEBUG [RS:2;jenkins-hbase5:38729] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK], DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK], DatanodeInfoWithStorage[127.0.0.1:46687,DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4,DISK]] 2023-07-12 15:23:05,445 INFO [RS:1;jenkins-hbase5:42707] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,42707,1689175384756/jenkins-hbase5.apache.org%2C42707%2C1689175384756.1689175385388 2023-07-12 15:23:05,445 DEBUG [RS:1;jenkins-hbase5:42707] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK], DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK], DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK]] 2023-07-12 15:23:05,451 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase5.apache.org,33173,1689175384809 2023-07-12 15:23:05,453 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:23:05,459 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK] 2023-07-12 15:23:05,460 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK] 2023-07-12 15:23:05,460 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK] 2023-07-12 15:23:05,463 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:53608, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 15:23:05,465 INFO [RS:0;jenkins-hbase5:35897] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,35897,1689175384731/jenkins-hbase5.apache.org%2C35897%2C1689175384731.1689175385409 2023-07-12 15:23:05,465 DEBUG [RS:0;jenkins-hbase5:35897] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK], DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK], DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK]] 2023-07-12 15:23:05,474 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-12 15:23:05,475 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:05,477 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C33173%2C1689175384809.meta, suffix=.meta, logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33173,1689175384809, archiveDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs, maxLogs=10 2023-07-12 15:23:05,507 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK] 2023-07-12 15:23:05,508 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK] 2023-07-12 15:23:05,507 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK] 2023-07-12 15:23:05,522 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33173,1689175384809/jenkins-hbase5.apache.org%2C33173%2C1689175384809.meta.1689175385478.meta 2023-07-12 15:23:05,523 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK], DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK], DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK]] 2023-07-12 15:23:05,523 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:05,523 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-12 15:23:05,523 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-12 15:23:05,524 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-12 15:23:05,524 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-12 15:23:05,524 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:05,525 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-12 15:23:05,525 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-12 15:23:05,527 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-12 15:23:05,528 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info 2023-07-12 15:23:05,528 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info 2023-07-12 15:23:05,529 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-12 15:23:05,530 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:05,530 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-12 15:23:05,531 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:23:05,531 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:23:05,532 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-12 15:23:05,533 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:05,533 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-12 15:23:05,534 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/table 2023-07-12 15:23:05,534 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/table 2023-07-12 15:23:05,534 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-12 15:23:05,535 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:05,537 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740 2023-07-12 15:23:05,541 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740 2023-07-12 15:23:05,544 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-12 15:23:05,547 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-12 15:23:05,549 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59727866, jitterRate=-0.10998544096946716}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-12 15:23:05,549 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-12 15:23:05,550 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689175385451 2023-07-12 15:23:05,560 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-12 15:23:05,560 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-12 15:23:05,561 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase5.apache.org,33173,1689175384809, state=OPEN 2023-07-12 15:23:05,564 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-12 15:23:05,564 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-12 15:23:05,568 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-12 15:23:05,568 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase5.apache.org,33173,1689175384809 in 269 msec 2023-07-12 15:23:05,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-12 15:23:05,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 441 msec 2023-07-12 15:23:05,576 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 524 msec 2023-07-12 15:23:05,577 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689175385577, completionTime=-1 2023-07-12 15:23:05,577 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=4; waited=0ms, expected min=4 server(s), max=4 server(s), master is running 2023-07-12 15:23:05,577 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:23:05,582 DEBUG [hconnection-0x7bf9bb43-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:05,587 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:53612, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:05,590 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=4 2023-07-12 15:23:05,590 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689175445590 2023-07-12 15:23:05,590 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689175505590 2023-07-12 15:23:05,590 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 12 msec 2023-07-12 15:23:05,600 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,45779,1689175384692-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,600 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,45779,1689175384692-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,600 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,45779,1689175384692-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,600 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase5:45779, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,600 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:05,600 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-12 15:23:05,600 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-12 15:23:05,602 DEBUG [master/jenkins-hbase5:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-12 15:23:05,602 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-12 15:23:05,604 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:23:05,606 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:23:05,615 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:05,616 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d empty. 2023-07-12 15:23:05,618 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:05,618 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-12 15:23:05,662 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-12 15:23:05,663 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => e7e73f29c14aa26ac2fc8049609eeb2d, NAME => 'hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp 2023-07-12 15:23:05,690 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:05,690 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing e7e73f29c14aa26ac2fc8049609eeb2d, disabling compactions & flushes 2023-07-12 15:23:05,690 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:05,690 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:05,690 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. after waiting 0 ms 2023-07-12 15:23:05,690 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:05,690 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:05,691 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for e7e73f29c14aa26ac2fc8049609eeb2d: 2023-07-12 15:23:05,696 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:23:05,698 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689175385698"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689175385698"}]},"ts":"1689175385698"} 2023-07-12 15:23:05,702 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:23:05,704 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:23:05,704 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689175385704"}]},"ts":"1689175385704"} 2023-07-12 15:23:05,706 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-12 15:23:05,713 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase5.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:23:05,714 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:23:05,714 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:23:05,714 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:23:05,714 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-12 15:23:05,714 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:23:05,714 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e7e73f29c14aa26ac2fc8049609eeb2d, ASSIGN}] 2023-07-12 15:23:05,718 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e7e73f29c14aa26ac2fc8049609eeb2d, ASSIGN 2023-07-12 15:23:05,719 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=e7e73f29c14aa26ac2fc8049609eeb2d, ASSIGN; state=OFFLINE, location=jenkins-hbase5.apache.org,42707,1689175384756; forceNewPlan=false, retain=false 2023-07-12 15:23:05,870 INFO [jenkins-hbase5:45779] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:23:05,871 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=e7e73f29c14aa26ac2fc8049609eeb2d, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:05,872 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689175385871"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175385871"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175385871"}]},"ts":"1689175385871"} 2023-07-12 15:23:05,874 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure e7e73f29c14aa26ac2fc8049609eeb2d, server=jenkins-hbase5.apache.org,42707,1689175384756}] 2023-07-12 15:23:06,028 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:06,029 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:23:06,031 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:35342, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 15:23:06,037 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:06,037 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e7e73f29c14aa26ac2fc8049609eeb2d, NAME => 'hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:06,037 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:06,037 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:06,038 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:06,038 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:06,039 INFO [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:06,042 DEBUG [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/info 2023-07-12 15:23:06,042 DEBUG [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/info 2023-07-12 15:23:06,044 INFO [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e7e73f29c14aa26ac2fc8049609eeb2d columnFamilyName info 2023-07-12 15:23:06,045 INFO [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] regionserver.HStore(310): Store=e7e73f29c14aa26ac2fc8049609eeb2d/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:06,048 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:06,049 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:06,054 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:06,058 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:23:06,059 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened e7e73f29c14aa26ac2fc8049609eeb2d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59722724, jitterRate=-0.11006206274032593}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:23:06,059 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for e7e73f29c14aa26ac2fc8049609eeb2d: 2023-07-12 15:23:06,060 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d., pid=6, masterSystemTime=1689175386028 2023-07-12 15:23:06,065 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:06,066 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:06,067 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=e7e73f29c14aa26ac2fc8049609eeb2d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:06,067 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689175386067"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175386067"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175386067"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175386067"}]},"ts":"1689175386067"} 2023-07-12 15:23:06,072 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-12 15:23:06,073 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure e7e73f29c14aa26ac2fc8049609eeb2d, server=jenkins-hbase5.apache.org,42707,1689175384756 in 196 msec 2023-07-12 15:23:06,076 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-12 15:23:06,077 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=e7e73f29c14aa26ac2fc8049609eeb2d, ASSIGN in 359 msec 2023-07-12 15:23:06,078 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:23:06,079 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689175386079"}]},"ts":"1689175386079"} 2023-07-12 15:23:06,081 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-12 15:23:06,084 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:23:06,087 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 484 msec 2023-07-12 15:23:06,103 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/namespace 2023-07-12 15:23:06,107 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/namespace 2023-07-12 15:23:06,107 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:06,111 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:06,112 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:35352, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:06,116 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-12 15:23:06,129 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-12 15:23:06,138 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 21 msec 2023-07-12 15:23:06,148 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:23:06,157 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-12 15:23:06,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-07-12 15:23:06,175 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-12 15:23:06,178 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-12 15:23:06,178 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.339sec 2023-07-12 15:23:06,178 INFO [master/jenkins-hbase5:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-12 15:23:06,178 INFO [master/jenkins-hbase5:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-12 15:23:06,178 INFO [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-12 15:23:06,180 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,45779,1689175384692-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-12 15:23:06,180 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,45779,1689175384692-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-12 15:23:06,183 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-12 15:23:06,229 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(139): Connect 0x4562c9d2 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:06,236 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@309340d6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:06,238 DEBUG [hconnection-0x9ff0290-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:06,239 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:53620, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:06,242 INFO [Listener at localhost/42735] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase5.apache.org,45779,1689175384692 2023-07-12 15:23:06,253 DEBUG [Listener at localhost/42735] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-12 15:23:06,255 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:34562, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-12 15:23:06,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.HMaster$4(2112): Client=jenkins//172.31.10.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-12 15:23:06,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-12 15:23:06,271 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:23:06,276 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:23:06,278 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:06,279 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932 empty. 2023-07-12 15:23:06,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.MasterRpcServices(700): Client=jenkins//172.31.10.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-12 15:23:06,280 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:06,280 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-12 15:23:06,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-12 15:23:06,331 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-12 15:23:06,333 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => e2a1d8488c64dcf2f22986998cf96932, NAME => 't1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp 2023-07-12 15:23:06,344 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:06,344 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing e2a1d8488c64dcf2f22986998cf96932, disabling compactions & flushes 2023-07-12 15:23:06,344 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:06,344 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:06,344 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. after waiting 0 ms 2023-07-12 15:23:06,344 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:06,344 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:06,344 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for e2a1d8488c64dcf2f22986998cf96932: 2023-07-12 15:23:06,348 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:23:06,350 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175386350"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689175386350"}]},"ts":"1689175386350"} 2023-07-12 15:23:06,352 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:23:06,353 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:23:06,354 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689175386354"}]},"ts":"1689175386354"} 2023-07-12 15:23:06,356 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-12 15:23:06,361 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase5.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:23:06,362 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:23:06,362 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:23:06,362 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:23:06,362 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=e2a1d8488c64dcf2f22986998cf96932, ASSIGN}] 2023-07-12 15:23:06,364 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=e2a1d8488c64dcf2f22986998cf96932, ASSIGN 2023-07-12 15:23:06,365 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=e2a1d8488c64dcf2f22986998cf96932, ASSIGN; state=OFFLINE, location=jenkins-hbase5.apache.org,36017,1689175380873; forceNewPlan=false, retain=false 2023-07-12 15:23:06,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-12 15:23:06,515 INFO [jenkins-hbase5:45903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:23:06,517 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=e2a1d8488c64dcf2f22986998cf96932, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:06,517 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175386517"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175386517"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175386517"}]},"ts":"1689175386517"} 2023-07-12 15:23:06,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure e2a1d8488c64dcf2f22986998cf96932, server=jenkins-hbase5.apache.org,36017,1689175380873}] 2023-07-12 15:23:06,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-12 15:23:06,673 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:06,674 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:23:06,675 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:58044, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 15:23:06,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:06,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e2a1d8488c64dcf2f22986998cf96932, NAME => 't1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:06,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:06,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:06,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:06,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:06,684 INFO [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:06,686 DEBUG [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/cf1 2023-07-12 15:23:06,686 DEBUG [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/cf1 2023-07-12 15:23:06,686 INFO [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e2a1d8488c64dcf2f22986998cf96932 columnFamilyName cf1 2023-07-12 15:23:06,687 INFO [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] regionserver.HStore(310): Store=e2a1d8488c64dcf2f22986998cf96932/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:06,687 INFO [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:06,689 DEBUG [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/norep 2023-07-12 15:23:06,689 DEBUG [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/norep 2023-07-12 15:23:06,689 INFO [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e2a1d8488c64dcf2f22986998cf96932 columnFamilyName norep 2023-07-12 15:23:06,690 INFO [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] regionserver.HStore(310): Store=e2a1d8488c64dcf2f22986998cf96932/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:06,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:06,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:06,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:23:06,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:06,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:23:06,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened e2a1d8488c64dcf2f22986998cf96932; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67618323, jitterRate=0.007591530680656433}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:23:06,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for e2a1d8488c64dcf2f22986998cf96932: 2023-07-12 15:23:06,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932., pid=11, masterSystemTime=1689175386673 2023-07-12 15:23:06,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:06,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:06,708 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=e2a1d8488c64dcf2f22986998cf96932, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:06,708 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175386708"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175386708"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175386708"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175386708"}]},"ts":"1689175386708"} 2023-07-12 15:23:06,714 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-12 15:23:06,714 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure e2a1d8488c64dcf2f22986998cf96932, server=jenkins-hbase5.apache.org,36017,1689175380873 in 192 msec 2023-07-12 15:23:06,716 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-12 15:23:06,717 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=e2a1d8488c64dcf2f22986998cf96932, ASSIGN in 352 msec 2023-07-12 15:23:06,718 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:23:06,719 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689175386718"}]},"ts":"1689175386718"} 2023-07-12 15:23:06,721 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-12 15:23:06,724 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:23:06,725 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 457 msec 2023-07-12 15:23:06,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-12 15:23:06,900 INFO [Listener at localhost/42735] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-12 15:23:06,902 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.HMaster$4(2112): Client=jenkins//172.31.10.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-12 15:23:06,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-12 15:23:06,906 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:23:06,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.MasterRpcServices(700): Client=jenkins//172.31.10.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-12 15:23:06,909 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:23:06,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-12 15:23:06,911 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:06,912 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b empty. 2023-07-12 15:23:06,912 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:06,912 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-12 15:23:06,931 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-12 15:23:06,932 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2c7136e581575f5b2280ddc3c686997b, NAME => 't2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.tmp 2023-07-12 15:23:06,950 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:06,950 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 2c7136e581575f5b2280ddc3c686997b, disabling compactions & flushes 2023-07-12 15:23:06,950 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:06,950 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:06,950 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. after waiting 0 ms 2023-07-12 15:23:06,950 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:06,950 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:06,950 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 2c7136e581575f5b2280ddc3c686997b: 2023-07-12 15:23:06,954 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:23:06,956 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175386956"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689175386956"}]},"ts":"1689175386956"} 2023-07-12 15:23:06,958 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:23:06,959 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:23:06,960 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689175386959"}]},"ts":"1689175386959"} 2023-07-12 15:23:06,962 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-12 15:23:06,966 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase5.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:23:06,968 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:23:06,968 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:23:06,968 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:23:06,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=2c7136e581575f5b2280ddc3c686997b, ASSIGN}] 2023-07-12 15:23:06,970 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=2c7136e581575f5b2280ddc3c686997b, ASSIGN 2023-07-12 15:23:06,972 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=2c7136e581575f5b2280ddc3c686997b, ASSIGN; state=OFFLINE, location=jenkins-hbase5.apache.org,36017,1689175380873; forceNewPlan=false, retain=false 2023-07-12 15:23:07,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-12 15:23:07,122 INFO [jenkins-hbase5:45903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:23:07,124 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=2c7136e581575f5b2280ddc3c686997b, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:07,124 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175387123"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175387123"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175387123"}]},"ts":"1689175387123"} 2023-07-12 15:23:07,126 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 2c7136e581575f5b2280ddc3c686997b, server=jenkins-hbase5.apache.org,36017,1689175380873}] 2023-07-12 15:23:07,211 INFO [jenkins-hbase5:36017Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:23:07,211 INFO [jenkins-hbase5:35567Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:23:07,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-12 15:23:07,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:07,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2c7136e581575f5b2280ddc3c686997b, NAME => 't2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:07,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:07,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:07,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:07,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:07,286 INFO [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:07,288 DEBUG [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/cf1 2023-07-12 15:23:07,288 DEBUG [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/cf1 2023-07-12 15:23:07,289 INFO [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c7136e581575f5b2280ddc3c686997b columnFamilyName cf1 2023-07-12 15:23:07,289 INFO [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] regionserver.HStore(310): Store=2c7136e581575f5b2280ddc3c686997b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:07,289 INFO [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:07,291 DEBUG [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/norep 2023-07-12 15:23:07,291 DEBUG [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/norep 2023-07-12 15:23:07,291 INFO [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c7136e581575f5b2280ddc3c686997b columnFamilyName norep 2023-07-12 15:23:07,292 INFO [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] regionserver.HStore(310): Store=2c7136e581575f5b2280ddc3c686997b/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:07,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:07,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:07,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:23:07,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:07,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:23:07,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 2c7136e581575f5b2280ddc3c686997b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70126714, jitterRate=0.04496946930885315}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:23:07,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 2c7136e581575f5b2280ddc3c686997b: 2023-07-12 15:23:07,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b., pid=14, masterSystemTime=1689175387279 2023-07-12 15:23:07,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:07,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:07,307 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=2c7136e581575f5b2280ddc3c686997b, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:07,307 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175387307"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175387307"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175387307"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175387307"}]},"ts":"1689175387307"} 2023-07-12 15:23:07,312 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-12 15:23:07,312 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 2c7136e581575f5b2280ddc3c686997b, server=jenkins-hbase5.apache.org,36017,1689175380873 in 184 msec 2023-07-12 15:23:07,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-12 15:23:07,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=2c7136e581575f5b2280ddc3c686997b, ASSIGN in 344 msec 2023-07-12 15:23:07,317 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:23:07,317 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689175387317"}]},"ts":"1689175387317"} 2023-07-12 15:23:07,318 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-12 15:23:07,321 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:23:07,323 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 419 msec 2023-07-12 15:23:07,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-12 15:23:07,515 INFO [Listener at localhost/42735] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-12 15:23:07,517 DEBUG [Listener at localhost/42735] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-12 15:23:07,519 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:59264, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-12 15:23:07,521 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45779] master.HMaster$4(2112): Client=jenkins//172.31.10.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-12 15:23:07,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45779] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-12 15:23:07,525 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:23:07,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45779] master.MasterRpcServices(700): Client=jenkins//172.31.10.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-12 15:23:07,527 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:23:07,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45779] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-12 15:23:07,529 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp/data/default/t1_syncup/19659d0923548261983bde115dd510a9 2023-07-12 15:23:07,530 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp/data/default/t1_syncup/19659d0923548261983bde115dd510a9 empty. 2023-07-12 15:23:07,530 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp/data/default/t1_syncup/19659d0923548261983bde115dd510a9 2023-07-12 15:23:07,530 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-12 15:23:07,552 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-12 15:23:07,553 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 19659d0923548261983bde115dd510a9, NAME => 't1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp 2023-07-12 15:23:07,600 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:07,600 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing 19659d0923548261983bde115dd510a9, disabling compactions & flushes 2023-07-12 15:23:07,600 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:07,600 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:07,600 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. after waiting 0 ms 2023-07-12 15:23:07,600 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:07,600 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:07,600 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 19659d0923548261983bde115dd510a9: 2023-07-12 15:23:07,604 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:23:07,606 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175387605"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689175387605"}]},"ts":"1689175387605"} 2023-07-12 15:23:07,608 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:23:07,609 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:23:07,609 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689175387609"}]},"ts":"1689175387609"} 2023-07-12 15:23:07,611 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-12 15:23:07,615 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase5.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:23:07,616 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:23:07,616 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:23:07,616 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:23:07,616 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-12 15:23:07,616 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:23:07,616 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=19659d0923548261983bde115dd510a9, ASSIGN}] 2023-07-12 15:23:07,618 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=19659d0923548261983bde115dd510a9, ASSIGN 2023-07-12 15:23:07,620 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=19659d0923548261983bde115dd510a9, ASSIGN; state=OFFLINE, location=jenkins-hbase5.apache.org,33173,1689175384809; forceNewPlan=false, retain=false 2023-07-12 15:23:07,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45779] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-12 15:23:07,770 INFO [jenkins-hbase5:45779] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:23:07,772 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=19659d0923548261983bde115dd510a9, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,33173,1689175384809 2023-07-12 15:23:07,772 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175387771"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175387771"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175387771"}]},"ts":"1689175387771"} 2023-07-12 15:23:07,774 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 19659d0923548261983bde115dd510a9, server=jenkins-hbase5.apache.org,33173,1689175384809}] 2023-07-12 15:23:07,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45779] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-12 15:23:07,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:07,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 19659d0923548261983bde115dd510a9, NAME => 't1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:07,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 19659d0923548261983bde115dd510a9 2023-07-12 15:23:07,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:07,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 19659d0923548261983bde115dd510a9 2023-07-12 15:23:07,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 19659d0923548261983bde115dd510a9 2023-07-12 15:23:07,937 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 19659d0923548261983bde115dd510a9 2023-07-12 15:23:07,938 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1 2023-07-12 15:23:07,938 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1 2023-07-12 15:23:07,939 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 19659d0923548261983bde115dd510a9 columnFamilyName cf1 2023-07-12 15:23:07,940 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(310): Store=19659d0923548261983bde115dd510a9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:07,940 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 19659d0923548261983bde115dd510a9 2023-07-12 15:23:07,941 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/norep 2023-07-12 15:23:07,941 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/norep 2023-07-12 15:23:07,942 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 19659d0923548261983bde115dd510a9 columnFamilyName norep 2023-07-12 15:23:07,942 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(310): Store=19659d0923548261983bde115dd510a9/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:07,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9 2023-07-12 15:23:07,944 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9 2023-07-12 15:23:07,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:23:07,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 19659d0923548261983bde115dd510a9 2023-07-12 15:23:07,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:23:07,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 19659d0923548261983bde115dd510a9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70100316, jitterRate=0.04457610845565796}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:23:07,950 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 19659d0923548261983bde115dd510a9: 2023-07-12 15:23:07,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9., pid=11, masterSystemTime=1689175387927 2023-07-12 15:23:07,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:07,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:07,954 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=19659d0923548261983bde115dd510a9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase5.apache.org,33173,1689175384809 2023-07-12 15:23:07,954 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175387954"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175387954"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175387954"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175387954"}]},"ts":"1689175387954"} 2023-07-12 15:23:07,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-12 15:23:07,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 19659d0923548261983bde115dd510a9, server=jenkins-hbase5.apache.org,33173,1689175384809 in 183 msec 2023-07-12 15:23:07,962 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-12 15:23:07,962 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=19659d0923548261983bde115dd510a9, ASSIGN in 344 msec 2023-07-12 15:23:07,963 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:23:07,963 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689175387963"}]},"ts":"1689175387963"} 2023-07-12 15:23:07,965 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-12 15:23:07,969 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:23:07,971 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 448 msec 2023-07-12 15:23:08,103 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-12 15:23:08,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45779] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-12 15:23:08,132 INFO [Listener at localhost/42735] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-12 15:23:08,133 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45779] master.HMaster$4(2112): Client=jenkins//172.31.10.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-12 15:23:08,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45779] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-12 15:23:08,138 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:23:08,139 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45779] master.MasterRpcServices(700): Client=jenkins//172.31.10.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-12 15:23:08,140 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:23:08,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45779] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-12 15:23:08,146 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:08,147 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604 empty. 2023-07-12 15:23:08,148 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:08,148 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-12 15:23:08,198 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-12 15:23:08,199 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 144fbe8ee3e556340eac4ab1f11a0604, NAME => 't2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.tmp 2023-07-12 15:23:08,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45779] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-12 15:23:08,265 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:08,267 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 144fbe8ee3e556340eac4ab1f11a0604, disabling compactions & flushes 2023-07-12 15:23:08,267 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:08,267 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:08,267 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. after waiting 0 ms 2023-07-12 15:23:08,267 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:08,267 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:08,267 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 144fbe8ee3e556340eac4ab1f11a0604: 2023-07-12 15:23:08,274 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:23:08,275 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175388275"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689175388275"}]},"ts":"1689175388275"} 2023-07-12 15:23:08,282 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:23:08,283 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:23:08,284 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689175388284"}]},"ts":"1689175388284"} 2023-07-12 15:23:08,286 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-12 15:23:08,291 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase5.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:23:08,293 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:23:08,293 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:23:08,293 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:23:08,293 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-12 15:23:08,293 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:23:08,293 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=144fbe8ee3e556340eac4ab1f11a0604, ASSIGN}] 2023-07-12 15:23:08,297 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=144fbe8ee3e556340eac4ab1f11a0604, ASSIGN 2023-07-12 15:23:08,300 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=144fbe8ee3e556340eac4ab1f11a0604, ASSIGN; state=OFFLINE, location=jenkins-hbase5.apache.org,42707,1689175384756; forceNewPlan=false, retain=false 2023-07-12 15:23:08,372 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-12 15:23:08,373 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-12 15:23:08,374 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-12 15:23:08,374 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-12 15:23:08,450 INFO [jenkins-hbase5:45779] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:23:08,452 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=144fbe8ee3e556340eac4ab1f11a0604, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:08,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45779] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-12 15:23:08,452 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175388452"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175388452"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175388452"}]},"ts":"1689175388452"} 2023-07-12 15:23:08,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 144fbe8ee3e556340eac4ab1f11a0604, server=jenkins-hbase5.apache.org,42707,1689175384756}] 2023-07-12 15:23:08,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:08,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 144fbe8ee3e556340eac4ab1f11a0604, NAME => 't2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:08,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:08,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:08,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:08,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:08,617 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:08,620 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1 2023-07-12 15:23:08,620 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1 2023-07-12 15:23:08,620 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 144fbe8ee3e556340eac4ab1f11a0604 columnFamilyName cf1 2023-07-12 15:23:08,621 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(310): Store=144fbe8ee3e556340eac4ab1f11a0604/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:08,621 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:08,623 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/norep 2023-07-12 15:23:08,623 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/norep 2023-07-12 15:23:08,624 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 144fbe8ee3e556340eac4ab1f11a0604 columnFamilyName norep 2023-07-12 15:23:08,624 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(310): Store=144fbe8ee3e556340eac4ab1f11a0604/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:08,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:08,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:08,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:23:08,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:08,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:23:08,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 144fbe8ee3e556340eac4ab1f11a0604; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66934311, jitterRate=-0.00260104238986969}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:23:08,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 144fbe8ee3e556340eac4ab1f11a0604: 2023-07-12 15:23:08,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604., pid=14, masterSystemTime=1689175388609 2023-07-12 15:23:08,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:08,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:08,643 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=144fbe8ee3e556340eac4ab1f11a0604, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:08,643 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175388643"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175388643"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175388643"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175388643"}]},"ts":"1689175388643"} 2023-07-12 15:23:08,651 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-12 15:23:08,651 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 144fbe8ee3e556340eac4ab1f11a0604, server=jenkins-hbase5.apache.org,42707,1689175384756 in 191 msec 2023-07-12 15:23:08,654 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-12 15:23:08,654 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=144fbe8ee3e556340eac4ab1f11a0604, ASSIGN in 358 msec 2023-07-12 15:23:08,655 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:23:08,655 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689175388655"}]},"ts":"1689175388655"} 2023-07-12 15:23:08,658 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-12 15:23:08,661 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:23:08,663 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 527 msec 2023-07-12 15:23:08,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45779] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-12 15:23:08,754 INFO [Listener at localhost/42735] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-12 15:23:08,755 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(139): Connect 0x1ffdd212 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:08,762 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7b49d3ec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:08,762 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(139): Connect 0x0fe1f1bf to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:08,774 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7e7dcf13, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:08,790 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.HMaster(3464): Client=jenkins//172.31.10.131 creating replication peer, id=1, config=clusterKey=127.0.0.1:64130:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false, state=ENABLED 2023-07-12 15:23:08,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:PRE_PEER_MODIFICATION; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-12 15:23:08,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-12 15:23:08,818 INFO [PEWorker-2] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-12 15:23:08,818 INFO [PEWorker-2] zookeeper.RecoverableZooKeeper(93): Process identifier=org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@3fb03c08check-peer-cluster-id connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:08,823 DEBUG [PEWorker-2-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@3fb03c08check-peer-cluster-id0x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:08,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@3fb03c08check-peer-cluster-id-0x101c5bd87ea0014 connected 2023-07-12 15:23:08,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-12 15:23:08,926 DEBUG [PEWorker-2-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@3fb03c08check-peer-cluster-id-0x101c5bd87ea0014, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:08,935 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:08,951 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=17, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-07-12 15:23:09,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-12 15:23:09,118 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35567] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=16 2023-07-12 15:23:09,118 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36017] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=17 2023-07-12 15:23:09,120 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-12 15:23:09,120 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-12 15:23:09,136 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-12 15:23:09,137 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-12 15:23:09,164 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-12 15:23:09,164 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-12 15:23:09,199 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=16 2023-07-12 15:23:09,199 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=17 2023-07-12 15:23:09,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] master.HMaster(3641): Remote procedure done, pid=17 2023-07-12 15:23:09,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45903] master.HMaster(3641): Remote procedure done, pid=16 2023-07-12 15:23:09,203 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45903] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase5.apache.org,36017,1689175380873 suceeded 2023-07-12 15:23:09,203 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45903] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase5.apache.org,35567,1689175380829 suceeded 2023-07-12 15:23:09,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 258 msec 2023-07-12 15:23:09,217 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-07-12 15:23:09,217 INFO [PEWorker-2] replication.AddPeerProcedure(102): Successfully added ENABLED peer 1, config clusterKey=127.0.0.1:64130:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false 2023-07-12 15:23:09,217 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 260 msec 2023-07-12 15:23:09,220 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure in 426 msec 2023-07-12 15:23:09,233 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x5ee1a156 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:09,240 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x584fd771 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:09,240 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@4c5779ce, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:09,246 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:09,250 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@87205ac, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:09,252 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:09,254 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:09,255 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101c5bd87ea0017 connected 2023-07-12 15:23:09,258 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:09,258 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=1) is replicating from cluster=098cb589-347a-4654-bf7c-f2a1c06d1dde to cluster=5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:23:09,259 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101c5bd87ea0018 connected 2023-07-12 15:23:09,261 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=1) is replicating from cluster=098cb589-347a-4654-bf7c-f2a1c06d1dde to cluster=5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:23:09,263 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase5.apache.org%2C36017%2C1689175380873 2023-07-12 15:23:09,263 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase5.apache.org%2C35567%2C1689175380829 2023-07-12 15:23:09,271 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-12 15:23:09,271 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-12 15:23:09,273 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase5.apache.org%2C35567%2C1689175380829,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase5.apache.org%2C35567%2C1689175380829 2023-07-12 15:23:09,273 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase5.apache.org%2C36017%2C1689175380873 2023-07-12 15:23:09,353 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 685 2023-07-12 15:23:09,353 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:09,353 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-12 15:23:09,353 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-12 15:23:09,354 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 685 2023-07-12 15:23:09,354 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:09,354 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase5.apache.org%2C35567%2C1689175380829,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,35567,1689175380829/jenkins-hbase5.apache.org%2C35567%2C1689175380829.1689175382331, lastWalPosition=600, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-12 15:23:09,354 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873/jenkins-hbase5.apache.org%2C36017%2C1689175380873.1689175382331, lastWalPosition=685, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-12 15:23:09,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45903] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-12 15:23:09,411 DEBUG [Listener at localhost/42735] replication.TestReplicationSyncUpTool(91): putAndReplicateRows 2023-07-12 15:23:09,414 DEBUG [hconnection-0x7cf5edca-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:09,416 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:57814, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:09,420 DEBUG [Listener at localhost/42735] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:09,422 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:58060, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:09,463 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:09,469 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 3185 2023-07-12 15:23:09,469 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceWALReader(162): Read 24 WAL entries eligible for replication 2023-07-12 15:23:09,469 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 3185 2023-07-12 15:23:09,469 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/e2a1d8488c64dcf2f22986998cf96932/4=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/5=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/6=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/7=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/8=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/9=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/10=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/11=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/12=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/13=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/14=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/15=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/16=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/17=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/18=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/19=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/20=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/21=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/22=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/23=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/24=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/25=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/26=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/27=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873/jenkins-hbase5.apache.org%2C36017%2C1689175380873.1689175382331, lastWalPosition=3185, nbRowKeys=24, nbHFiles=0, heapSize=4992, lastSeqIds={}, endOfFile=false,usedBufferSize=4992] 2023-07-12 15:23:09,484 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:23:09,492 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:57338, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-12 15:23:09,497 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35897] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-12 15:23:09,498 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35897] zookeeper.ReadOnlyZKClient(139): Connect 0x0906f31e to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:09,517 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35897] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@46f466f9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:09,527 DEBUG [hconnection-0x585b37fb-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:09,528 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:43390, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-12 15:23:09,572 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:09,585 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=35897] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-12 15:23:09,593 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 10535 2023-07-12 15:23:09,593 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceWALReader(162): Read 70 WAL entries eligible for replication 2023-07-12 15:23:09,594 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/e2a1d8488c64dcf2f22986998cf96932/28=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/29=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/30=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/31=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/32=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/33=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/34=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/35=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/36=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/37=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/38=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/39=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/40=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/41=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/42=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/43=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/44=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/45=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/46=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/47=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/48=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/49=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/50=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/51=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/52=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/53=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/54=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/55=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/56=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/57=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/58=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/59=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/60=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/61=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/62=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/63=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/64=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/65=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/66=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/67=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/68=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/69=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/70=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/71=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/72=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/73=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/74=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/75=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/76=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/77=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/78=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/79=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/80=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/81=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/82=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/83=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/84=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/85=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/86=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/87=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/88=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/89=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/90=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/91=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/92=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/93=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/94=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/95=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/96=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/97=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873/jenkins-hbase5.apache.org%2C36017%2C1689175380873.1689175382331, lastWalPosition=10535, nbRowKeys=70, nbHFiles=0, heapSize=14560, lastSeqIds={}, endOfFile=false,usedBufferSize=14560] 2023-07-12 15:23:09,595 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 10535 2023-07-12 15:23:09,605 DEBUG [SinkThread-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:23:09,607 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:39442, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-12 15:23:09,610 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42707] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-12 15:23:09,611 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42707] zookeeper.ReadOnlyZKClient(139): Connect 0x454b3f30 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:09,624 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42707] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@412f6516, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:09,627 DEBUG [hconnection-0x74c4de55-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:09,628 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:43394, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-07-12 15:23:09,641 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42707] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-12 15:23:09,682 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:09,723 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 18291 2023-07-12 15:23:09,723 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceWALReader(162): Read 73 WAL entries eligible for replication 2023-07-12 15:23:09,724 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/e2a1d8488c64dcf2f22986998cf96932/98=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/99=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/100=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/101=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/102=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/103=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/4=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/5=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/6=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/7=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/8=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/9=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/10=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/11=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/12=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/13=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/14=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/15=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/16=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/17=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/18=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/19=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/20=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/21=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/22=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/23=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/24=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/25=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/26=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/27=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/28=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/29=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/30=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/31=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/32=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/33=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/34=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/35=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/36=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/37=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/38=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/39=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/40=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/41=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/42=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/43=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/44=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/45=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/46=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/47=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/48=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/49=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/50=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/51=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/52=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/53=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/54=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/55=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/56=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/57=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/58=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/59=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/60=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/61=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/62=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/63=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/64=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/65=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/66=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/67=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/68=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/69=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/70=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873/jenkins-hbase5.apache.org%2C36017%2C1689175380873.1689175382331, lastWalPosition=18291, nbRowKeys=73, nbHFiles=0, heapSize=15184, lastSeqIds={}, endOfFile=false,usedBufferSize=15184] 2023-07-12 15:23:09,725 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 18291 2023-07-12 15:23:09,726 DEBUG [SinkThread-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:23:09,736 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:48956, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-12 15:23:09,739 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38729] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-12 15:23:09,740 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38729] zookeeper.ReadOnlyZKClient(139): Connect 0x2a54d768 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:09,740 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-12 15:23:09,741 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-12 15:23:09,749 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@24932c52, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:09,755 DEBUG [hconnection-0x20acbc18-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:09,756 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:43396, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-12 15:23:09,767 DEBUG [hconnection-0x20acbc18-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:09,769 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:39444, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-12 15:23:09,784 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38729] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-12 15:23:09,796 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:09,842 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 24110 2023-07-12 15:23:09,842 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceWALReader(162): Read 55 WAL entries eligible for replication 2023-07-12 15:23:09,842 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/2c7136e581575f5b2280ddc3c686997b/71=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/72=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/73=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/74=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/75=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/76=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/77=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/78=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/79=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/80=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/81=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/82=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/83=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/84=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/85=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/86=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/87=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/88=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/89=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/90=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/91=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/92=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/93=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/94=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/95=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/96=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/97=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/98=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/99=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/100=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/101=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/102=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/103=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/104=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/105=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/106=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/107=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/108=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/109=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/110=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/111=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/112=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/113=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/114=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/115=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/116=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/117=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/118=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/119=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/120=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/121=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/122=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/123=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/124=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/125=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873/jenkins-hbase5.apache.org%2C36017%2C1689175380873.1689175382331, lastWalPosition=24110, nbRowKeys=55, nbHFiles=0, heapSize=11440, lastSeqIds={}, endOfFile=false,usedBufferSize=11440] 2023-07-12 15:23:09,843 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 24110 2023-07-12 15:23:09,846 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38729] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-12 15:23:09,858 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38729] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-12 15:23:09,908 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:09,958 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:09,958 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceWALReader(162): Read 78 WAL entries eligible for replication 2023-07-12 15:23:09,958 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:09,958 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/2c7136e581575f5b2280ddc3c686997b/126=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/127=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/128=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/129=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/130=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/131=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/132=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/133=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/134=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/135=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/136=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/137=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/138=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/139=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/140=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/141=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/142=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/143=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/144=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/145=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/146=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/147=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/148=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/149=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/150=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/151=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/152=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/153=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/154=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/155=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/156=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/157=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/158=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/159=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/160=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/161=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/162=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/163=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/164=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/165=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/166=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/167=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/168=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/169=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/170=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/171=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/172=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/173=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/174=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/175=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/176=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/177=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/178=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/179=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/180=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/181=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/182=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/183=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/184=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/185=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/186=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/187=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/188=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/189=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/190=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/191=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/192=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/193=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/194=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/195=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/196=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/197=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/198=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/199=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/200=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/201=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/202=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/203=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873/jenkins-hbase5.apache.org%2C36017%2C1689175380873.1689175382331, lastWalPosition=32644, nbRowKeys=78, nbHFiles=0, heapSize=16224, lastSeqIds={}, endOfFile=false,usedBufferSize=16224] 2023-07-12 15:23:09,962 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42707] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-12 15:23:09,965 DEBUG [hconnection-0x74c4de55-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:09,967 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:39450, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-07-12 15:23:09,971 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=42707] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-12 15:23:10,016 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:10,066 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:10,124 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:10,173 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:10,235 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:10,277 INFO [jenkins-hbase5:33173Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:23:10,279 INFO [jenkins-hbase5:38729Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:23:10,281 INFO [jenkins-hbase5:38729Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 128 2023-07-12 15:23:10,281 INFO [jenkins-hbase5:42707Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:23:10,281 INFO [jenkins-hbase5:42707Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 148 2023-07-12 15:23:10,283 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:10,292 INFO [jenkins-hbase5:35897Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:23:10,292 INFO [jenkins-hbase5:35897Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 24 2023-07-12 15:23:10,347 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:10,396 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:10,448 DEBUG [hconnection-0x4b1ea583-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:10,450 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:43410, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:10,455 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:10,471 DEBUG [hconnection-0x4b1ea583-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:10,476 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:39454, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:10,482 DEBUG [Listener at localhost/42735] replication.TestReplicationSyncUpTool(145): mimicSyncUpAfterDelete 2023-07-12 15:23:10,482 INFO [Listener at localhost/42735] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-12 15:23:10,483 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4562c9d2 to 127.0.0.1:64130 2023-07-12 15:23:10,483 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:10,484 DEBUG [Listener at localhost/42735] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-12 15:23:10,484 DEBUG [Listener at localhost/42735] util.JVMClusterUtil(257): Found active master hash=462729024, stopped=false 2023-07-12 15:23:10,484 INFO [Listener at localhost/42735] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase5.apache.org,45779,1689175384692 2023-07-12 15:23:10,488 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-12 15:23:10,488 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-12 15:23:10,488 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-12 15:23:10,488 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-12 15:23:10,488 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-12 15:23:10,488 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:10,488 INFO [Listener at localhost/42735] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:23:10,488 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x51755a69 to 127.0.0.1:64130 2023-07-12 15:23:10,490 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:10,491 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:10,491 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:10,491 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:10,491 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:10,491 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:10,492 INFO [Listener at localhost/42735] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase5.apache.org,35897,1689175384731' ***** 2023-07-12 15:23:10,492 INFO [Listener at localhost/42735] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 15:23:10,492 INFO [Listener at localhost/42735] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase5.apache.org,42707,1689175384756' ***** 2023-07-12 15:23:10,492 INFO [RS:0;jenkins-hbase5:35897] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 15:23:10,493 INFO [RS:0;jenkins-hbase5:35897] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 15:23:10,492 INFO [Listener at localhost/42735] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 15:23:10,493 INFO [RS:0;jenkins-hbase5:35897] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 15:23:10,493 INFO [Listener at localhost/42735] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase5.apache.org,38729,1689175384783' ***** 2023-07-12 15:23:10,493 INFO [RS:0;jenkins-hbase5:35897] regionserver.HRegionServer(1144): stopping server jenkins-hbase5.apache.org,35897,1689175384731 2023-07-12 15:23:10,493 INFO [Listener at localhost/42735] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 15:23:10,493 INFO [RS:1;jenkins-hbase5:42707] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 15:23:10,493 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 15:23:10,493 INFO [RS:2;jenkins-hbase5:38729] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 15:23:10,493 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 15:23:10,495 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 15:23:10,493 INFO [RS:1;jenkins-hbase5:42707] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 15:23:10,493 INFO [Listener at localhost/42735] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase5.apache.org,33173,1689175384809' ***** 2023-07-12 15:23:10,493 DEBUG [RS:0;jenkins-hbase5:35897] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x24128b82 to 127.0.0.1:64130 2023-07-12 15:23:10,497 INFO [RS:2;jenkins-hbase5:38729] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 15:23:10,495 INFO [RS:1;jenkins-hbase5:42707] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 15:23:10,498 INFO [RS:2;jenkins-hbase5:38729] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 15:23:10,498 INFO [Listener at localhost/42735] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 15:23:10,498 INFO [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(3305): Received CLOSE for 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:10,498 INFO [RS:3;jenkins-hbase5:33173] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 15:23:10,498 INFO [RS:3;jenkins-hbase5:33173] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 15:23:10,498 INFO [RS:3;jenkins-hbase5:33173] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 15:23:10,498 INFO [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(3305): Received CLOSE for 19659d0923548261983bde115dd510a9 2023-07-12 15:23:10,498 DEBUG [RS:0;jenkins-hbase5:35897] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:10,499 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 15:23:10,499 INFO [RS:0;jenkins-hbase5:35897] regionserver.HRegionServer(1170): stopping server jenkins-hbase5.apache.org,35897,1689175384731; all regions closed. 2023-07-12 15:23:10,498 INFO [RS:2;jenkins-hbase5:38729] regionserver.HRegionServer(1144): stopping server jenkins-hbase5.apache.org,38729,1689175384783 2023-07-12 15:23:10,500 DEBUG [RS:2;jenkins-hbase5:38729] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7d9c25a0 to 127.0.0.1:64130 2023-07-12 15:23:10,500 DEBUG [RS:2;jenkins-hbase5:38729] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:10,500 INFO [RS:2;jenkins-hbase5:38729] regionserver.HRegionServer(1170): stopping server jenkins-hbase5.apache.org,38729,1689175384783; all regions closed. 2023-07-12 15:23:10,500 INFO [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(3305): Received CLOSE for e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:10,500 INFO [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(1144): stopping server jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:10,500 DEBUG [RS:1;jenkins-hbase5:42707] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7bf78aee to 127.0.0.1:64130 2023-07-12 15:23:10,500 DEBUG [RS:1;jenkins-hbase5:42707] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:10,501 INFO [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-12 15:23:10,501 DEBUG [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(1478): Online Regions={144fbe8ee3e556340eac4ab1f11a0604=t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604., e7e73f29c14aa26ac2fc8049609eeb2d=hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.} 2023-07-12 15:23:10,507 INFO [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(1144): stopping server jenkins-hbase5.apache.org,33173,1689175384809 2023-07-12 15:23:10,507 DEBUG [RS:3;jenkins-hbase5:33173] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x21119e79 to 127.0.0.1:64130 2023-07-12 15:23:10,507 DEBUG [RS:3;jenkins-hbase5:33173] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:10,507 INFO [RS:3;jenkins-hbase5:33173] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 15:23:10,507 INFO [RS:3;jenkins-hbase5:33173] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 15:23:10,507 INFO [RS:3;jenkins-hbase5:33173] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 15:23:10,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 144fbe8ee3e556340eac4ab1f11a0604, disabling compactions & flushes 2023-07-12 15:23:10,508 INFO [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-12 15:23:10,510 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:10,510 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:10,510 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. after waiting 0 ms 2023-07-12 15:23:10,510 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:10,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 19659d0923548261983bde115dd510a9, disabling compactions & flushes 2023-07-12 15:23:10,511 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:10,511 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:10,511 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. after waiting 0 ms 2023-07-12 15:23:10,511 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:10,511 INFO [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-12 15:23:10,511 DEBUG [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(1478): Online Regions={19659d0923548261983bde115dd510a9=t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9., 1588230740=hbase:meta,,1.1588230740} 2023-07-12 15:23:10,511 DEBUG [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(1504): Waiting on 144fbe8ee3e556340eac4ab1f11a0604, e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:10,512 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-12 15:23:10,511 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing 19659d0923548261983bde115dd510a9 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-12 15:23:10,511 DEBUG [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(1504): Waiting on 1588230740, 19659d0923548261983bde115dd510a9 2023-07-12 15:23:10,511 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing 144fbe8ee3e556340eac4ab1f11a0604 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-12 15:23:10,512 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-12 15:23:10,513 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-12 15:23:10,513 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-12 15:23:10,513 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-12 15:23:10,513 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.65 KB heapSize=7.06 KB 2023-07-12 15:23:10,514 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:10,542 DEBUG [RS:0;jenkins-hbase5:35897] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs 2023-07-12 15:23:10,542 INFO [RS:0;jenkins-hbase5:35897] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase5.apache.org%2C35897%2C1689175384731:(num 1689175385409) 2023-07-12 15:23:10,542 DEBUG [RS:0;jenkins-hbase5:35897] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:10,542 DEBUG [RS:2;jenkins-hbase5:38729] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs 2023-07-12 15:23:10,542 INFO [RS:2;jenkins-hbase5:38729] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase5.apache.org%2C38729%2C1689175384783:(num 1689175385388) 2023-07-12 15:23:10,542 DEBUG [RS:2;jenkins-hbase5:38729] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:10,542 INFO [RS:2;jenkins-hbase5:38729] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:10,543 INFO [RS:2;jenkins-hbase5:38729] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase5:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-12 15:23:10,543 INFO [RS:2;jenkins-hbase5:38729] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 15:23:10,543 INFO [RS:2;jenkins-hbase5:38729] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 15:23:10,543 INFO [regionserver/jenkins-hbase5:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:23:10,542 INFO [RS:0;jenkins-hbase5:35897] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:10,543 INFO [RS:2;jenkins-hbase5:38729] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 15:23:10,545 INFO [RS:0;jenkins-hbase5:35897] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase5:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-12 15:23:10,546 INFO [RS:0;jenkins-hbase5:35897] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 15:23:10,546 INFO [regionserver/jenkins-hbase5:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:23:10,546 INFO [RS:0;jenkins-hbase5:35897] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 15:23:10,546 INFO [RS:0;jenkins-hbase5:35897] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 15:23:10,546 DEBUG [RS:2;jenkins-hbase5:38729] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2a54d768 to 127.0.0.1:64130 2023-07-12 15:23:10,546 DEBUG [RS:2;jenkins-hbase5:38729] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:10,547 DEBUG [RS:0;jenkins-hbase5:35897] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0906f31e to 127.0.0.1:64130 2023-07-12 15:23:10,547 DEBUG [RS:0;jenkins-hbase5:35897] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:10,548 INFO [RS:2;jenkins-hbase5:38729] ipc.NettyRpcServer(158): Stopping server on /172.31.10.131:38729 2023-07-12 15:23:10,548 INFO [RS:0;jenkins-hbase5:35897] ipc.NettyRpcServer(158): Stopping server on /172.31.10.131:35897 2023-07-12 15:23:10,555 INFO [regionserver/jenkins-hbase5:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:10,555 INFO [regionserver/jenkins-hbase5:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:10,559 INFO [regionserver/jenkins-hbase5:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:10,564 INFO [regionserver/jenkins-hbase5:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:10,569 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:10,597 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase5.apache.org,35897,1689175384731 2023-07-12 15:23:10,597 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase5.apache.org,35897,1689175384731 2023-07-12 15:23:10,597 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase5.apache.org,35897,1689175384731 2023-07-12 15:23:10,597 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101c5bd87ea0017, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:10,597 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:10,598 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase5.apache.org,38729,1689175384783 2023-07-12 15:23:10,597 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase5.apache.org,35897,1689175384731 2023-07-12 15:23:10,598 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:10,598 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase5.apache.org,38729,1689175384783 2023-07-12 15:23:10,597 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-12 15:23:10,597 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:10,597 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:10,597 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:10,598 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase5.apache.org,38729,1689175384783 2023-07-12 15:23:10,598 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase5.apache.org,38729,1689175384783 2023-07-12 15:23:10,638 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/.tmp/info/f9013e31cdf14f9ca5ad1cff1f13869f 2023-07-12 15:23:10,638 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/.tmp/cf1/90e6bd84cbb748218903eb87a1f792f7 2023-07-12 15:23:10,638 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/.tmp/cf1/82858d2ae5f44600b94c571e38fb835c 2023-07-12 15:23:10,639 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:10,683 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:10,693 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/.tmp/cf1/82858d2ae5f44600b94c571e38fb835c as hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1/82858d2ae5f44600b94c571e38fb835c 2023-07-12 15:23:10,696 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/.tmp/cf1/90e6bd84cbb748218903eb87a1f792f7 as hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1/90e6bd84cbb748218903eb87a1f792f7 2023-07-12 15:23:10,708 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1/82858d2ae5f44600b94c571e38fb835c, entries=100, sequenceid=7, filesize=8.3 K 2023-07-12 15:23:10,712 DEBUG [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(1504): Waiting on 144fbe8ee3e556340eac4ab1f11a0604, e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:10,713 DEBUG [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(1504): Waiting on 1588230740, 19659d0923548261983bde115dd510a9 2023-07-12 15:23:10,713 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 19659d0923548261983bde115dd510a9 in 202ms, sequenceid=7, compaction requested=false 2023-07-12 15:23:10,715 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1/90e6bd84cbb748218903eb87a1f792f7, entries=200, sequenceid=7, filesize=12.2 K 2023-07-12 15:23:10,720 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 144fbe8ee3e556340eac4ab1f11a0604 in 209ms, sequenceid=7, compaction requested=false 2023-07-12 15:23:10,730 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/.tmp/table/351115f8a9734cc98bccaabf45019170 2023-07-12 15:23:10,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-07-12 15:23:10,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-07-12 15:23:10,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:10,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 144fbe8ee3e556340eac4ab1f11a0604: 2023-07-12 15:23:10,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:10,739 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing e7e73f29c14aa26ac2fc8049609eeb2d, disabling compactions & flushes 2023-07-12 15:23:10,739 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:10,739 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:10,739 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. after waiting 0 ms 2023-07-12 15:23:10,739 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:10,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:10,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing e7e73f29c14aa26ac2fc8049609eeb2d 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-12 15:23:10,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 19659d0923548261983bde115dd510a9: 2023-07-12 15:23:10,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:10,748 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/.tmp/info/f9013e31cdf14f9ca5ad1cff1f13869f as hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info/f9013e31cdf14f9ca5ad1cff1f13869f 2023-07-12 15:23:10,756 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:10,759 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info/f9013e31cdf14f9ca5ad1cff1f13869f, entries=30, sequenceid=19, filesize=8.1 K 2023-07-12 15:23:10,760 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/.tmp/table/351115f8a9734cc98bccaabf45019170 as hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/table/351115f8a9734cc98bccaabf45019170 2023-07-12 15:23:10,770 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/table/351115f8a9734cc98bccaabf45019170, entries=6, sequenceid=19, filesize=4.8 K 2023-07-12 15:23:10,771 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.65 KB/3738, heapSize ~6.78 KB/6944, currentSize=0 B/0 for 1588230740 in 258ms, sequenceid=19, compaction requested=false 2023-07-12 15:23:10,772 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/.tmp/info/98bda8b543444aaf919145966712fca0 2023-07-12 15:23:10,781 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-12 15:23:10,782 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-12 15:23:10,783 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-12 15:23:10,783 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-12 15:23:10,783 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-12 15:23:10,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/.tmp/info/98bda8b543444aaf919145966712fca0 as hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/info/98bda8b543444aaf919145966712fca0 2023-07-12 15:23:10,790 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:10,792 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/info/98bda8b543444aaf919145966712fca0, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:23:10,793 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for e7e73f29c14aa26ac2fc8049609eeb2d in 53ms, sequenceid=6, compaction requested=false 2023-07-12 15:23:10,801 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase5.apache.org,38729,1689175384783] 2023-07-12 15:23:10,802 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase5.apache.org,38729,1689175384783; numProcessing=1 2023-07-12 15:23:10,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-12 15:23:10,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:10,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for e7e73f29c14aa26ac2fc8049609eeb2d: 2023-07-12 15:23:10,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:10,864 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:10,899 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:10,906 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase5.apache.org,38729,1689175384783 already deleted, retry=false 2023-07-12 15:23:10,906 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase5.apache.org,38729,1689175384783 expired; onlineServers=3 2023-07-12 15:23:10,906 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase5.apache.org,35897,1689175384731] 2023-07-12 15:23:10,906 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase5.apache.org,35897,1689175384731; numProcessing=2 2023-07-12 15:23:10,907 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase5.apache.org,35897,1689175384731 already deleted, retry=false 2023-07-12 15:23:10,907 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase5.apache.org,35897,1689175384731 expired; onlineServers=2 2023-07-12 15:23:10,913 INFO [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(1170): stopping server jenkins-hbase5.apache.org,42707,1689175384756; all regions closed. 2023-07-12 15:23:10,913 INFO [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(1170): stopping server jenkins-hbase5.apache.org,33173,1689175384809; all regions closed. 2023-07-12 15:23:10,922 DEBUG [RS:1;jenkins-hbase5:42707] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs 2023-07-12 15:23:10,922 DEBUG [RS:3;jenkins-hbase5:33173] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs 2023-07-12 15:23:10,922 INFO [RS:3;jenkins-hbase5:33173] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase5.apache.org%2C33173%2C1689175384809.meta:.meta(num 1689175385478) 2023-07-12 15:23:10,922 INFO [RS:1;jenkins-hbase5:42707] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase5.apache.org%2C42707%2C1689175384756:(num 1689175385388) 2023-07-12 15:23:10,922 DEBUG [RS:1;jenkins-hbase5:42707] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:10,922 INFO [RS:1;jenkins-hbase5:42707] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:10,922 INFO [RS:1;jenkins-hbase5:42707] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase5:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-12 15:23:10,922 INFO [RS:1;jenkins-hbase5:42707] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 15:23:10,922 INFO [regionserver/jenkins-hbase5:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:23:10,922 INFO [RS:1;jenkins-hbase5:42707] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 15:23:10,923 INFO [RS:1;jenkins-hbase5:42707] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 15:23:10,923 DEBUG [RS:1;jenkins-hbase5:42707] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x454b3f30 to 127.0.0.1:64130 2023-07-12 15:23:10,923 DEBUG [RS:1;jenkins-hbase5:42707] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:10,924 INFO [RS:1;jenkins-hbase5:42707] ipc.NettyRpcServer(158): Stopping server on /172.31.10.131:42707 2023-07-12 15:23:10,927 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:10,927 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101c5bd87ea0017, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:10,927 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:10,927 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-12 15:23:10,927 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase5.apache.org,42707,1689175384756 2023-07-12 15:23:10,928 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase5.apache.org,42707,1689175384756] 2023-07-12 15:23:10,928 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase5.apache.org,42707,1689175384756; numProcessing=3 2023-07-12 15:23:10,929 DEBUG [RS:3;jenkins-hbase5:33173] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs 2023-07-12 15:23:10,929 INFO [RS:3;jenkins-hbase5:33173] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase5.apache.org%2C33173%2C1689175384809:(num 1689175385386) 2023-07-12 15:23:10,929 DEBUG [RS:3;jenkins-hbase5:33173] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:10,929 INFO [RS:3;jenkins-hbase5:33173] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:10,930 INFO [RS:3;jenkins-hbase5:33173] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase5:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-12 15:23:10,930 INFO [regionserver/jenkins-hbase5:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:23:10,931 INFO [RS:3;jenkins-hbase5:33173] ipc.NettyRpcServer(158): Stopping server on /172.31.10.131:33173 2023-07-12 15:23:10,932 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase5.apache.org,42707,1689175384756 already deleted, retry=false 2023-07-12 15:23:10,932 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase5.apache.org,42707,1689175384756 expired; onlineServers=1 2023-07-12 15:23:10,933 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase5.apache.org,33173,1689175384809 2023-07-12 15:23:10,933 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:10,933 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101c5bd87ea0017, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:10,933 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-12 15:23:10,934 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase5.apache.org,33173,1689175384809] 2023-07-12 15:23:10,934 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase5.apache.org,33173,1689175384809; numProcessing=4 2023-07-12 15:23:10,936 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase5.apache.org,33173,1689175384809 already deleted, retry=false 2023-07-12 15:23:10,936 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase5.apache.org,33173,1689175384809 expired; onlineServers=0 2023-07-12 15:23:10,936 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase5.apache.org,45779,1689175384692' ***** 2023-07-12 15:23:10,936 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-12 15:23:10,937 DEBUG [M:0;jenkins-hbase5:45779] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1fab28ea, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase5.apache.org/172.31.10.131:0 2023-07-12 15:23:10,937 INFO [M:0;jenkins-hbase5:45779] regionserver.HRegionServer(1144): stopping server jenkins-hbase5.apache.org,45779,1689175384692 2023-07-12 15:23:10,937 INFO [M:0;jenkins-hbase5:45779] regionserver.HRegionServer(1170): stopping server jenkins-hbase5.apache.org,45779,1689175384692; all regions closed. 2023-07-12 15:23:10,937 DEBUG [M:0;jenkins-hbase5:45779] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:10,937 DEBUG [M:0;jenkins-hbase5:45779] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-12 15:23:10,937 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-12 15:23:10,937 DEBUG [M:0;jenkins-hbase5:45779] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-12 15:23:10,937 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175385059] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175385059,5,FailOnTimeoutGroup] 2023-07-12 15:23:10,937 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175385059] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175385059,5,FailOnTimeoutGroup] 2023-07-12 15:23:10,938 INFO [M:0;jenkins-hbase5:45779] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-12 15:23:10,940 INFO [M:0;jenkins-hbase5:45779] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-12 15:23:10,940 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-12 15:23:10,940 INFO [M:0;jenkins-hbase5:45779] hbase.ChoreService(369): Chore service for: master/jenkins-hbase5:0 had [] on shutdown 2023-07-12 15:23:10,940 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:10,940 DEBUG [M:0;jenkins-hbase5:45779] master.HMaster(1512): Stopping service threads 2023-07-12 15:23:10,940 INFO [M:0;jenkins-hbase5:45779] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-12 15:23:10,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-12 15:23:10,941 ERROR [M:0;jenkins-hbase5:45779] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-07-12 15:23:10,942 INFO [M:0;jenkins-hbase5:45779] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:23:10,942 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-12 15:23:10,942 DEBUG [M:0;jenkins-hbase5:45779] zookeeper.ZKUtil(398): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-12 15:23:10,942 WARN [M:0;jenkins-hbase5:45779] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-12 15:23:10,942 INFO [M:0;jenkins-hbase5:45779] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:23:10,943 INFO [M:0;jenkins-hbase5:45779] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-12 15:23:10,943 DEBUG [M:0;jenkins-hbase5:45779] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-12 15:23:10,943 INFO [M:0;jenkins-hbase5:45779] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:10,943 DEBUG [M:0;jenkins-hbase5:45779] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:10,944 DEBUG [M:0;jenkins-hbase5:45779] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 1 ms 2023-07-12 15:23:10,944 DEBUG [M:0;jenkins-hbase5:45779] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:10,944 INFO [M:0;jenkins-hbase5:45779] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=55.39 KB heapSize=65.18 KB 2023-07-12 15:23:10,959 INFO [M:0;jenkins-hbase5:45779] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=55.39 KB at sequenceid=116 (bloomFilter=true), to=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/4b68c55f96324a40933fdb475c17211a 2023-07-12 15:23:10,966 DEBUG [M:0;jenkins-hbase5:45779] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/4b68c55f96324a40933fdb475c17211a as hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/4b68c55f96324a40933fdb475c17211a 2023-07-12 15:23:10,969 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:10,973 INFO [M:0;jenkins-hbase5:45779] regionserver.HStore(1080): Added hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/4b68c55f96324a40933fdb475c17211a, entries=14, sequenceid=116, filesize=8.1 K 2023-07-12 15:23:10,974 INFO [M:0;jenkins-hbase5:45779] regionserver.HRegion(2948): Finished flush of dataSize ~55.39 KB/56720, heapSize ~65.16 KB/66728, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 30ms, sequenceid=116, compaction requested=false 2023-07-12 15:23:10,976 INFO [M:0;jenkins-hbase5:45779] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:10,976 DEBUG [M:0;jenkins-hbase5:45779] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:23:10,981 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:23:10,981 INFO [M:0;jenkins-hbase5:45779] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-12 15:23:10,982 INFO [M:0;jenkins-hbase5:45779] ipc.NettyRpcServer(158): Stopping server on /172.31.10.131:45779 2023-07-12 15:23:10,984 DEBUG [M:0;jenkins-hbase5:45779] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase5.apache.org,45779,1689175384692 already deleted, retry=false 2023-07-12 15:23:10,998 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:10,998 INFO [RS:2;jenkins-hbase5:38729] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase5.apache.org,38729,1689175384783; zookeeper connection closed. 2023-07-12 15:23:10,998 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:38729-0x101c5bd87ea000a, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:10,999 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@387e1560] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@387e1560 2023-07-12 15:23:11,007 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:11,075 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:11,098 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:11,098 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:45779-0x101c5bd87ea0007, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:11,098 INFO [M:0;jenkins-hbase5:45779] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase5.apache.org,45779,1689175384692; zookeeper connection closed. 2023-07-12 15:23:11,112 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:11,181 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:11,198 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:11,199 INFO [RS:3;jenkins-hbase5:33173] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase5.apache.org,33173,1689175384809; zookeeper connection closed. 2023-07-12 15:23:11,199 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33173-0x101c5bd87ea000b, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:11,200 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@243aed30] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@243aed30 2023-07-12 15:23:11,218 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:11,287 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:11,299 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:11,299 INFO [RS:1;jenkins-hbase5:42707] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase5.apache.org,42707,1689175384756; zookeeper connection closed. 2023-07-12 15:23:11,299 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:42707-0x101c5bd87ea0009, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:11,300 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@53373c7e] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@53373c7e 2023-07-12 15:23:11,324 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:11,394 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:11,437 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-12 15:23:11,499 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:11,500 INFO [RS:0;jenkins-hbase5:35897] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase5.apache.org,35897,1689175384731; zookeeper connection closed. 2023-07-12 15:23:11,500 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:35897-0x101c5bd87ea0008, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:11,501 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3b3d65c5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3b3d65c5 2023-07-12 15:23:11,502 INFO [Listener at localhost/42735] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 4 regionserver(s) complete 2023-07-12 15:23:11,505 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 32644 2023-07-12 15:23:11,526 INFO [Listener at localhost/42735] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-12 15:23:11,526 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5de352d4 to 127.0.0.1:64130 2023-07-12 15:23:11,526 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:11,526 DEBUG [Listener at localhost/42735] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-12 15:23:11,526 DEBUG [Listener at localhost/42735] util.JVMClusterUtil(257): Found active master hash=17542254, stopped=false 2023-07-12 15:23:11,526 INFO [Listener at localhost/42735] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase5.apache.org,45903,1689175379564 2023-07-12 15:23:11,529 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:35567-0x101c5bd87ea0001, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-12 15:23:11,529 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-12 15:23:11,529 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-12 15:23:11,529 INFO [Listener at localhost/42735] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:23:11,529 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:11,531 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7c3827af to 127.0.0.1:64130 2023-07-12 15:23:11,531 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35567-0x101c5bd87ea0001, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-12 15:23:11,531 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:11,532 INFO [Listener at localhost/42735] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase5.apache.org,35567,1689175380829' ***** 2023-07-12 15:23:11,532 INFO [Listener at localhost/42735] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 15:23:11,532 INFO [Listener at localhost/42735] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase5.apache.org,36017,1689175380873' ***** 2023-07-12 15:23:11,532 INFO [Listener at localhost/42735] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 15:23:11,532 INFO [RS:1;jenkins-hbase5:36017] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 15:23:11,532 INFO [RS:1;jenkins-hbase5:36017] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 15:23:11,532 INFO [RS:1;jenkins-hbase5:36017] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 15:23:11,533 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-12 15:23:11,533 INFO [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(3305): Received CLOSE for 2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:11,533 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-12 15:23:11,533 INFO [RS:0;jenkins-hbase5:35567] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 15:23:11,533 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 15:23:11,534 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 15:23:11,534 INFO [RS:0;jenkins-hbase5:35567] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 15:23:11,534 INFO [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(3305): Received CLOSE for e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:11,534 INFO [RS:0;jenkins-hbase5:35567] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 15:23:11,534 INFO [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1144): stopping server jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:11,536 INFO [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(3305): Received CLOSE for 2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:11,536 DEBUG [RS:1;jenkins-hbase5:36017] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x711237df to 127.0.0.1:64130 2023-07-12 15:23:11,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 2c7136e581575f5b2280ddc3c686997b, disabling compactions & flushes 2023-07-12 15:23:11,536 DEBUG [RS:1;jenkins-hbase5:36017] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:11,537 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:11,537 INFO [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-12 15:23:11,537 INFO [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(1144): stopping server jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:11,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:11,537 DEBUG [RS:0;jenkins-hbase5:35567] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x139497cc to 127.0.0.1:64130 2023-07-12 15:23:11,537 DEBUG [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1478): Online Regions={2c7136e581575f5b2280ddc3c686997b=t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b., e2a1d8488c64dcf2f22986998cf96932=t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932.} 2023-07-12 15:23:11,538 DEBUG [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1504): Waiting on 2c7136e581575f5b2280ddc3c686997b, e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:11,538 DEBUG [RS:0;jenkins-hbase5:35567] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:11,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. after waiting 0 ms 2023-07-12 15:23:11,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:11,538 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing 2c7136e581575f5b2280ddc3c686997b 2/2 column families, dataSize=12.70 KB heapSize=42.80 KB 2023-07-12 15:23:11,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 2f5120c49287770b2e8c9c01ffb12e34, disabling compactions & flushes 2023-07-12 15:23:11,539 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:11,539 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:11,538 INFO [RS:0;jenkins-hbase5:35567] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 15:23:11,539 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. after waiting 0 ms 2023-07-12 15:23:11,539 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:11,539 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing 2f5120c49287770b2e8c9c01ffb12e34 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-12 15:23:11,539 INFO [RS:0;jenkins-hbase5:35567] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 15:23:11,539 INFO [RS:0;jenkins-hbase5:35567] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 15:23:11,540 INFO [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-12 15:23:11,540 INFO [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-12 15:23:11,540 DEBUG [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(1478): Online Regions={2f5120c49287770b2e8c9c01ffb12e34=hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34., 1588230740=hbase:meta,,1.1588230740} 2023-07-12 15:23:11,541 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-12 15:23:11,541 DEBUG [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(1504): Waiting on 1588230740, 2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:11,541 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-12 15:23:11,541 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-12 15:23:11,541 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-12 15:23:11,541 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-12 15:23:11,542 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.87 KB heapSize=7.44 KB 2023-07-12 15:23:11,545 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 845 2023-07-12 15:23:11,545 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-12 15:23:11,545 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase5.apache.org%2C35567%2C1689175380829,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,35567,1689175380829/jenkins-hbase5.apache.org%2C35567%2C1689175380829.1689175382331, lastWalPosition=845, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-12 15:23:11,545 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 845 2023-07-12 15:23:11,573 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/.tmp/info/08bc8a0183c64c54b0ce393f29c1440b 2023-07-12 15:23:11,579 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34/.tmp/info/5b1005ad38d547d98cb6dd0a418765c2 2023-07-12 15:23:11,584 INFO [regionserver/jenkins-hbase5:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:11,585 INFO [regionserver/jenkins-hbase5:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:11,587 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.74 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/.tmp/cf1/318232eb904e4b49acd994e031633b58 2023-07-12 15:23:11,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34/.tmp/info/5b1005ad38d547d98cb6dd0a418765c2 as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34/info/5b1005ad38d547d98cb6dd0a418765c2 2023-07-12 15:23:11,636 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 318232eb904e4b49acd994e031633b58 2023-07-12 15:23:11,640 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 42881 2023-07-12 15:23:11,640 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceWALReader(162): Read 2 WAL entries eligible for replication 2023-07-12 15:23:11,641 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/e2a1d8488c64dcf2f22986998cf96932/105=[#edits: 50 = ],4120}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873/jenkins-hbase5.apache.org%2C36017%2C1689175380873.1689175382331, lastWalPosition=42881, nbRowKeys=150, nbHFiles=0, heapSize=12240, lastSeqIds={}, endOfFile=false,usedBufferSize=12240] 2023-07-12 15:23:11,641 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 42881 2023-07-12 15:23:11,646 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34/info/5b1005ad38d547d98cb6dd0a418765c2, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:23:11,647 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 2f5120c49287770b2e8c9c01ffb12e34 in 108ms, sequenceid=6, compaction requested=false 2023-07-12 15:23:11,696 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 1124 2023-07-12 15:23:11,696 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-12 15:23:11,697 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 1124 2023-07-12 15:23:11,726 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/.tmp/rep_barrier/3699d38d4fe144b4b4117dfd4f0b5c51 2023-07-12 15:23:11,738 DEBUG [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1504): Waiting on 2c7136e581575f5b2280ddc3c686997b, e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:11,741 DEBUG [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(1504): Waiting on 1588230740, 2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:11,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-12 15:23:11,776 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 42881 2023-07-12 15:23:11,778 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:11,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 2f5120c49287770b2e8c9c01ffb12e34: 2023-07-12 15:23:11,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:11,784 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.96 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/.tmp/norep/6bcf7203f4f84b4a97b28d7d0ba402e4 2023-07-12 15:23:11,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6bcf7203f4f84b4a97b28d7d0ba402e4 2023-07-12 15:23:11,793 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/.tmp/cf1/318232eb904e4b49acd994e031633b58 as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/cf1/318232eb904e4b49acd994e031633b58 2023-07-12 15:23:11,802 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 318232eb904e4b49acd994e031633b58 2023-07-12 15:23:11,802 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/cf1/318232eb904e4b49acd994e031633b58, entries=200, sequenceid=206, filesize=11.9 K 2023-07-12 15:23:11,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/.tmp/norep/6bcf7203f4f84b4a97b28d7d0ba402e4 as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/norep/6bcf7203f4f84b4a97b28d7d0ba402e4 2023-07-12 15:23:11,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6bcf7203f4f84b4a97b28d7d0ba402e4 2023-07-12 15:23:11,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/norep/6bcf7203f4f84b4a97b28d7d0ba402e4, entries=101, sequenceid=206, filesize=8.3 K 2023-07-12 15:23:11,832 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~12.70 KB/13001, heapSize ~42.77 KB/43800, currentSize=0 B/0 for 2c7136e581575f5b2280ddc3c686997b in 294ms, sequenceid=206, compaction requested=false 2023-07-12 15:23:11,834 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-12 15:23:11,834 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-12 15:23:11,846 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/.tmp/table/452d4cba21c1453fa96f77780c0f84eb 2023-07-12 15:23:11,854 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/recovered.edits/209.seqid, newMaxSeqId=209, maxSeqId=1 2023-07-12 15:23:11,856 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:11,856 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 2c7136e581575f5b2280ddc3c686997b: 2023-07-12 15:23:11,857 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:11,858 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing e2a1d8488c64dcf2f22986998cf96932, disabling compactions & flushes 2023-07-12 15:23:11,858 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:11,858 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:11,858 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. after waiting 0 ms 2023-07-12 15:23:11,858 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:11,858 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing e2a1d8488c64dcf2f22986998cf96932 2/2 column families, dataSize=6.25 KB heapSize=21.71 KB 2023-07-12 15:23:11,862 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/.tmp/info/08bc8a0183c64c54b0ce393f29c1440b as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/info/08bc8a0183c64c54b0ce393f29c1440b 2023-07-12 15:23:11,873 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/info/08bc8a0183c64c54b0ce393f29c1440b, entries=30, sequenceid=19, filesize=8.1 K 2023-07-12 15:23:11,875 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/.tmp/rep_barrier/3699d38d4fe144b4b4117dfd4f0b5c51 as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/rep_barrier/3699d38d4fe144b4b4117dfd4f0b5c51 2023-07-12 15:23:11,885 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 43737 2023-07-12 15:23:11,885 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-12 15:23:11,885 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 43737 2023-07-12 15:23:11,886 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/rep_barrier/3699d38d4fe144b4b4117dfd4f0b5c51, entries=2, sequenceid=19, filesize=4.9 K 2023-07-12 15:23:11,888 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/.tmp/table/452d4cba21c1453fa96f77780c0f84eb as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/table/452d4cba21c1453fa96f77780c0f84eb 2023-07-12 15:23:11,888 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.76 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/.tmp/cf1/3749472cc82744ac9a6eaf3cb152ec28 2023-07-12 15:23:11,894 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3749472cc82744ac9a6eaf3cb152ec28 2023-07-12 15:23:11,895 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/table/452d4cba21c1453fa96f77780c0f84eb, entries=6, sequenceid=19, filesize=4.8 K 2023-07-12 15:23:11,896 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.87 KB/3964, heapSize ~7.39 KB/7568, currentSize=0 B/0 for 1588230740 in 355ms, sequenceid=19, compaction requested=false 2023-07-12 15:23:11,916 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-12 15:23:11,917 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-12 15:23:11,924 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-12 15:23:11,924 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-12 15:23:11,924 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-12 15:23:11,933 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.50 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/.tmp/norep/161ea13628764952839a9efd2f670468 2023-07-12 15:23:11,938 DEBUG [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1504): Waiting on e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:11,940 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 161ea13628764952839a9efd2f670468 2023-07-12 15:23:11,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/.tmp/cf1/3749472cc82744ac9a6eaf3cb152ec28 as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/cf1/3749472cc82744ac9a6eaf3cb152ec28 2023-07-12 15:23:11,941 INFO [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(1170): stopping server jenkins-hbase5.apache.org,35567,1689175380829; all regions closed. 2023-07-12 15:23:11,954 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3749472cc82744ac9a6eaf3cb152ec28 2023-07-12 15:23:11,955 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/cf1/3749472cc82744ac9a6eaf3cb152ec28, entries=100, sequenceid=106, filesize=8.2 K 2023-07-12 15:23:11,955 DEBUG [RS:0;jenkins-hbase5:35567] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs 2023-07-12 15:23:11,955 INFO [RS:0;jenkins-hbase5:35567] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase5.apache.org%2C35567%2C1689175380829.meta:.meta(num 1689175382467) 2023-07-12 15:23:11,956 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/.tmp/norep/161ea13628764952839a9efd2f670468 as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/norep/161ea13628764952839a9efd2f670468 2023-07-12 15:23:11,976 DEBUG [RS:0;jenkins-hbase5:35567] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs 2023-07-12 15:23:11,976 INFO [RS:0;jenkins-hbase5:35567] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase5.apache.org%2C35567%2C1689175380829:(num 1689175382331) 2023-07-12 15:23:11,976 DEBUG [RS:0;jenkins-hbase5:35567] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:11,976 INFO [RS:0;jenkins-hbase5:35567] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:11,977 INFO [RS:0;jenkins-hbase5:35567] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase5:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-12 15:23:11,977 INFO [regionserver/jenkins-hbase5:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:23:11,978 INFO [RS:0;jenkins-hbase5:35567] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-12 15:23:11,980 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 161ea13628764952839a9efd2f670468 2023-07-12 15:23:11,980 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/norep/161ea13628764952839a9efd2f670468, entries=51, sequenceid=106, filesize=6.5 K 2023-07-12 15:23:11,981 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~6.25 KB/6401, heapSize ~21.68 KB/22200, currentSize=0 B/0 for e2a1d8488c64dcf2f22986998cf96932 in 123ms, sequenceid=106, compaction requested=false 2023-07-12 15:23:11,993 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] wal.ProtobufLogReader(420): EOF at position 44348 2023-07-12 15:23:11,993 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-12 15:23:11,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/recovered.edits/109.seqid, newMaxSeqId=109, maxSeqId=1 2023-07-12 15:23:11,997 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:11,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for e2a1d8488c64dcf2f22986998cf96932: 2023-07-12 15:23:11,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:12,082 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101c5bd87ea0018, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:12,082 DEBUG [RS:0;jenkins-hbase5:35567] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x584fd771 to 127.0.0.1:64130 2023-07-12 15:23:12,085 DEBUG [RS:0;jenkins-hbase5:35567] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:12,139 INFO [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1170): stopping server jenkins-hbase5.apache.org,36017,1689175380873; all regions closed. 2023-07-12 15:23:12,146 DEBUG [RS:1;jenkins-hbase5:36017] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs 2023-07-12 15:23:12,146 INFO [RS:1;jenkins-hbase5:36017] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase5.apache.org%2C36017%2C1689175380873:(num 1689175382331) 2023-07-12 15:23:12,146 DEBUG [RS:1;jenkins-hbase5:36017] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:12,147 INFO [RS:1;jenkins-hbase5:36017] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:12,147 INFO [RS:1;jenkins-hbase5:36017] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase5:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-12 15:23:12,147 INFO [RS:1;jenkins-hbase5:36017] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 15:23:12,147 INFO [regionserver/jenkins-hbase5:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:23:12,147 INFO [RS:1;jenkins-hbase5:36017] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 15:23:12,148 INFO [RS:1;jenkins-hbase5:36017] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 15:23:12,148 INFO [RS:1;jenkins-hbase5:36017] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-12 15:23:12,185 INFO [RS:0;jenkins-hbase5:35567] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase5.apache.org%2C35567%2C1689175380829,1 terminated 2023-07-12 15:23:12,187 INFO [RS:0;jenkins-hbase5:35567] ipc.NettyRpcServer(158): Stopping server on /172.31.10.131:35567 2023-07-12 15:23:12,189 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:35567-0x101c5bd87ea0001, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:12,189 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-12 15:23:12,189 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:35567-0x101c5bd87ea0001, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-12 15:23:12,189 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:12,190 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-12 15:23:12,190 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase5.apache.org,35567,1689175380829] 2023-07-12 15:23:12,190 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase5.apache.org,35567,1689175380829; numProcessing=1 2023-07-12 15:23:12,193 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase5.apache.org,35567,1689175380829 already deleted, retry=false 2023-07-12 15:23:12,193 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase5.apache.org,35567,1689175380829 expired; onlineServers=1 2023-07-12 15:23:12,211 INFO [jenkins-hbase5:36017Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=12240B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 300, current progress: walGroup [jenkins-hbase5.apache.org%2C36017%2C1689175380873]: currently replicating from: hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873/jenkins-hbase5.apache.org%2C36017%2C1689175380873.1689175382331 at position: 32644 2023-07-12 15:23:12,288 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101c5bd87ea0017, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:12,288 DEBUG [RS:1;jenkins-hbase5:36017] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5ee1a156 to 127.0.0.1:64130 2023-07-12 15:23:12,288 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101c5bd87ea0017, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:12,288 DEBUG [RS:1;jenkins-hbase5:36017] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:12,388 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:35567-0x101c5bd87ea0001, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:12,388 INFO [RS:0;jenkins-hbase5:35567] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase5.apache.org,35567,1689175380829; zookeeper connection closed. 2023-07-12 15:23:12,388 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:35567-0x101c5bd87ea0001, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:12,389 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@75c69a57] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@75c69a57 2023-07-12 15:23:12,389 INFO [RS:1;jenkins-hbase5:36017] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase5:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1 terminated 2023-07-12 15:23:12,391 INFO [RS:1;jenkins-hbase5:36017] ipc.NettyRpcServer(158): Stopping server on /172.31.10.131:36017 2023-07-12 15:23:12,394 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-12 15:23:12,394 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:12,396 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase5.apache.org,36017,1689175380873] 2023-07-12 15:23:12,396 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase5.apache.org,36017,1689175380873; numProcessing=2 2023-07-12 15:23:12,397 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase5.apache.org,36017,1689175380873 already deleted, retry=false 2023-07-12 15:23:12,398 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase5.apache.org,36017,1689175380873 expired; onlineServers=0 2023-07-12 15:23:12,398 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase5.apache.org,45903,1689175379564' ***** 2023-07-12 15:23:12,398 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-12 15:23:12,399 DEBUG [M:0;jenkins-hbase5:45903] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4036a655, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase5.apache.org/172.31.10.131:0 2023-07-12 15:23:12,399 INFO [M:0;jenkins-hbase5:45903] regionserver.HRegionServer(1144): stopping server jenkins-hbase5.apache.org,45903,1689175379564 2023-07-12 15:23:12,399 INFO [M:0;jenkins-hbase5:45903] regionserver.HRegionServer(1170): stopping server jenkins-hbase5.apache.org,45903,1689175379564; all regions closed. 2023-07-12 15:23:12,399 DEBUG [M:0;jenkins-hbase5:45903] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:12,399 DEBUG [M:0;jenkins-hbase5:45903] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-12 15:23:12,399 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-12 15:23:12,400 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175381984] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175381984,5,FailOnTimeoutGroup] 2023-07-12 15:23:12,399 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175381984] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175381984,5,FailOnTimeoutGroup] 2023-07-12 15:23:12,399 DEBUG [M:0;jenkins-hbase5:45903] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-12 15:23:12,401 INFO [M:0;jenkins-hbase5:45903] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-12 15:23:12,401 INFO [M:0;jenkins-hbase5:45903] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-12 15:23:12,401 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-12 15:23:12,401 INFO [M:0;jenkins-hbase5:45903] hbase.ChoreService(369): Chore service for: master/jenkins-hbase5:0 had [] on shutdown 2023-07-12 15:23:12,401 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:12,402 DEBUG [M:0;jenkins-hbase5:45903] master.HMaster(1512): Stopping service threads 2023-07-12 15:23:12,402 INFO [M:0;jenkins-hbase5:45903] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-12 15:23:12,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-12 15:23:12,403 ERROR [M:0;jenkins-hbase5:45903] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] 2023-07-12 15:23:12,403 INFO [M:0;jenkins-hbase5:45903] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:23:12,403 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-12 15:23:12,403 DEBUG [M:0;jenkins-hbase5:45903] zookeeper.ZKUtil(398): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-12 15:23:12,403 WARN [M:0;jenkins-hbase5:45903] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-12 15:23:12,403 INFO [M:0;jenkins-hbase5:45903] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:23:12,404 INFO [M:0;jenkins-hbase5:45903] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-12 15:23:12,405 DEBUG [M:0;jenkins-hbase5:45903] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-12 15:23:12,405 INFO [M:0;jenkins-hbase5:45903] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:12,405 DEBUG [M:0;jenkins-hbase5:45903] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:12,405 DEBUG [M:0;jenkins-hbase5:45903] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-12 15:23:12,405 DEBUG [M:0;jenkins-hbase5:45903] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:12,405 INFO [M:0;jenkins-hbase5:45903] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=61.03 KB heapSize=72.60 KB 2023-07-12 15:23:12,420 INFO [M:0;jenkins-hbase5:45903] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=61.03 KB at sequenceid=136 (bloomFilter=true), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e3ba367d06984e51a93df5b25e050654 2023-07-12 15:23:12,426 DEBUG [M:0;jenkins-hbase5:45903] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e3ba367d06984e51a93df5b25e050654 as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e3ba367d06984e51a93df5b25e050654 2023-07-12 15:23:12,434 INFO [M:0;jenkins-hbase5:45903] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e3ba367d06984e51a93df5b25e050654, entries=17, sequenceid=136, filesize=8.5 K 2023-07-12 15:23:12,435 INFO [M:0;jenkins-hbase5:45903] regionserver.HRegion(2948): Finished flush of dataSize ~61.03 KB/62496, heapSize ~72.59 KB/74328, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 30ms, sequenceid=136, compaction requested=false 2023-07-12 15:23:12,437 INFO [M:0;jenkins-hbase5:45903] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:12,437 DEBUG [M:0;jenkins-hbase5:45903] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:23:12,441 INFO [M:0;jenkins-hbase5:45903] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-12 15:23:12,441 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:23:12,442 INFO [M:0;jenkins-hbase5:45903] ipc.NettyRpcServer(158): Stopping server on /172.31.10.131:45903 2023-07-12 15:23:12,443 DEBUG [M:0;jenkins-hbase5:45903] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase5.apache.org,45903,1689175379564 already deleted, retry=false 2023-07-12 15:23:12,589 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:12,589 INFO [M:0;jenkins-hbase5:45903] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase5.apache.org,45903,1689175379564; zookeeper connection closed. 2023-07-12 15:23:12,589 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): master:45903-0x101c5bd87ea0000, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:12,689 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:12,689 INFO [RS:1;jenkins-hbase5:36017] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase5.apache.org,36017,1689175380873; zookeeper connection closed. 2023-07-12 15:23:12,689 DEBUG [Listener at localhost/35683-EventThread] zookeeper.ZKWatcher(600): regionserver:36017-0x101c5bd87ea0002, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:12,689 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@50a5a642] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@50a5a642 2023-07-12 15:23:12,689 INFO [Listener at localhost/42735] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 2 regionserver(s) complete 2023-07-12 15:23:12,691 INFO [Listener at localhost/42735] client.ConnectionUtils(127): master/jenkins-hbase5:0 server-side Connection retries=18 2023-07-12 15:23:12,691 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:12,691 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:12,691 INFO [Listener at localhost/42735] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:23:12,691 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:12,691 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:23:12,692 INFO [Listener at localhost/42735] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:23:12,692 INFO [Listener at localhost/42735] ipc.NettyRpcServer(120): Bind to /172.31.10.131:39489 2023-07-12 15:23:12,693 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:12,694 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:12,695 INFO [Listener at localhost/42735] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39489 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:12,699 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:394890x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:12,700 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:39489-0x101c5bd87ea001c connected 2023-07-12 15:23:12,702 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-12 15:23:12,702 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:12,703 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-12 15:23:12,703 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39489 2023-07-12 15:23:12,703 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39489 2023-07-12 15:23:12,704 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39489 2023-07-12 15:23:12,704 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39489 2023-07-12 15:23:12,704 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39489 2023-07-12 15:23:12,704 INFO [Listener at localhost/42735] master.HMaster(444): hbase.rootdir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143, hbase.cluster.distributed=false 2023-07-12 15:23:12,705 DEBUG [pool-440-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-12 15:23:12,719 INFO [Listener at localhost/42735] client.ConnectionUtils(127): regionserver/jenkins-hbase5:0 server-side Connection retries=18 2023-07-12 15:23:12,719 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:12,719 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:12,719 INFO [Listener at localhost/42735] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:23:12,719 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:12,719 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:23:12,720 INFO [Listener at localhost/42735] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:23:12,720 INFO [Listener at localhost/42735] ipc.NettyRpcServer(120): Bind to /172.31.10.131:40223 2023-07-12 15:23:12,721 INFO [Listener at localhost/42735] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 15:23:12,722 DEBUG [Listener at localhost/42735] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 15:23:12,723 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:12,725 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:12,726 INFO [Listener at localhost/42735] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40223 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:12,730 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:402230x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:12,731 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:402230x0, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-12 15:23:12,732 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:40223-0x101c5bd87ea001d connected 2023-07-12 15:23:12,732 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:40223-0x101c5bd87ea001d, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:12,733 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:40223-0x101c5bd87ea001d, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-12 15:23:12,733 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40223 2023-07-12 15:23:12,733 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40223 2023-07-12 15:23:12,734 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40223 2023-07-12 15:23:12,734 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40223 2023-07-12 15:23:12,735 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40223 2023-07-12 15:23:12,737 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase5.apache.org,39489,1689175392691 2023-07-12 15:23:12,738 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-12 15:23:12,739 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase5.apache.org,39489,1689175392691 2023-07-12 15:23:12,741 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:40223-0x101c5bd87ea001d, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-12 15:23:12,741 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-12 15:23:12,741 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:12,742 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-12 15:23:12,743 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase5.apache.org,39489,1689175392691 from backup master directory 2023-07-12 15:23:12,743 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-12 15:23:12,744 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase5.apache.org,39489,1689175392691 2023-07-12 15:23:12,744 WARN [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:23:12,744 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-12 15:23:12,744 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase5.apache.org,39489,1689175392691 2023-07-12 15:23:12,759 INFO [master/jenkins-hbase5:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:13,186 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x606ee338 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:13,190 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7c98b96c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:13,190 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-12 15:23:13,191 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-12 15:23:13,191 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:13,199 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,45779,1689175384692 to hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,45779,1689175384692-dead as it is dead 2023-07-12 15:23:13,200 INFO [master/jenkins-hbase5:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,45779,1689175384692-dead/jenkins-hbase5.apache.org%2C45779%2C1689175384692.1689175384950 2023-07-12 15:23:13,205 INFO [master/jenkins-hbase5:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,45779,1689175384692-dead/jenkins-hbase5.apache.org%2C45779%2C1689175384692.1689175384950 after 5ms 2023-07-12 15:23:13,206 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,45779,1689175384692-dead/jenkins-hbase5.apache.org%2C45779%2C1689175384692.1689175384950 to hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase5.apache.org%2C45779%2C1689175384692.1689175384950 2023-07-12 15:23:13,206 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,45779,1689175384692-dead 2023-07-12 15:23:13,206 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,39489,1689175392691 2023-07-12 15:23:13,209 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C39489%2C1689175392691, suffix=, logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,39489,1689175392691, archiveDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/oldWALs, maxLogs=10 2023-07-12 15:23:13,223 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK] 2023-07-12 15:23:13,224 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46687,DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4,DISK] 2023-07-12 15:23:13,224 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK] 2023-07-12 15:23:13,230 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,39489,1689175392691/jenkins-hbase5.apache.org%2C39489%2C1689175392691.1689175393209 2023-07-12 15:23:13,230 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK], DatanodeInfoWithStorage[127.0.0.1:46687,DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4,DISK], DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK]] 2023-07-12 15:23:13,230 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:13,231 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:13,231 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:13,231 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:13,232 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:13,233 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-12 15:23:13,233 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-12 15:23:13,240 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/4b68c55f96324a40933fdb475c17211a 2023-07-12 15:23:13,241 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:13,242 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-12 15:23:13,242 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase5.apache.org%2C45779%2C1689175384692.1689175384950 2023-07-12 15:23:13,248 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 131, firstSequenceIdInLog=3, maxSequenceIdInLog=118, path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase5.apache.org%2C45779%2C1689175384692.1689175384950 2023-07-12 15:23:13,249 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase5.apache.org%2C45779%2C1689175384692.1689175384950 2023-07-12 15:23:13,252 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:13,254 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=1 2023-07-12 15:23:13,255 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=119; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72984287, jitterRate=0.08755062520503998}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:23:13,255 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:23:13,255 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-12 15:23:13,256 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-12 15:23:13,256 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-12 15:23:13,257 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:23:13,257 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:23:13,271 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:23:13,272 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:23:13,272 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:23:13,273 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:23:13,273 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-12 15:23:13,273 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-12 15:23:13,274 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 16 msec 2023-07-12 15:23:13,274 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-12 15:23:13,275 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:23:13,275 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase5.apache.org,33173,1689175384809, table=hbase:meta, region=1588230740 2023-07-12 15:23:13,277 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 4 possibly 'live' servers, and 0 'splitting'. 2023-07-12 15:23:13,279 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase5.apache.org,33173,1689175384809 already deleted, retry=false 2023-07-12 15:23:13,279 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase5.apache.org,33173,1689175384809 on jenkins-hbase5.apache.org,39489,1689175392691 2023-07-12 15:23:13,286 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase5.apache.org,33173,1689175384809, splitWal=true, meta=true 2023-07-12 15:23:13,286 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=15 for jenkins-hbase5.apache.org,33173,1689175384809 (carryingMeta=true) jenkins-hbase5.apache.org,33173,1689175384809/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@71fdf522[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-12 15:23:13,288 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase5.apache.org,38729,1689175384783 already deleted, retry=false 2023-07-12 15:23:13,288 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase5.apache.org,38729,1689175384783 on jenkins-hbase5.apache.org,39489,1689175392691 2023-07-12 15:23:13,289 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=16, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase5.apache.org,38729,1689175384783, splitWal=true, meta=false 2023-07-12 15:23:13,289 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=16 for jenkins-hbase5.apache.org,38729,1689175384783 (carryingMeta=false) jenkins-hbase5.apache.org,38729,1689175384783/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@5ec95ebe[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-12 15:23:13,290 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase5.apache.org,42707,1689175384756 already deleted, retry=false 2023-07-12 15:23:13,290 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase5.apache.org,42707,1689175384756 on jenkins-hbase5.apache.org,39489,1689175392691 2023-07-12 15:23:13,291 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase5.apache.org,42707,1689175384756, splitWal=true, meta=false 2023-07-12 15:23:13,291 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=17 for jenkins-hbase5.apache.org,42707,1689175384756 (carryingMeta=false) jenkins-hbase5.apache.org,42707,1689175384756/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@4083f4a0[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-12 15:23:13,292 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase5.apache.org,35897,1689175384731 already deleted, retry=false 2023-07-12 15:23:13,292 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase5.apache.org,35897,1689175384731 on jenkins-hbase5.apache.org,39489,1689175392691 2023-07-12 15:23:13,293 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase5.apache.org,35897,1689175384731, splitWal=true, meta=false 2023-07-12 15:23:13,293 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase5.apache.org,35897,1689175384731 (carryingMeta=false) jenkins-hbase5.apache.org,35897,1689175384731/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@460f90e7[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-12 15:23:13,303 INFO [master/jenkins-hbase5:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-12 15:23:13,304 INFO [master/jenkins-hbase5:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-12 15:23:13,304 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-12 15:23:13,304 INFO [master/jenkins-hbase5:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-12 15:23:13,304 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-12 15:23:13,305 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-12 15:23:13,305 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-12 15:23:13,306 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-12 15:23:13,308 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:40223-0x101c5bd87ea001d, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-12 15:23:13,308 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-12 15:23:13,309 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:13,309 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase5.apache.org,39489,1689175392691, sessionid=0x101c5bd87ea001c, setting cluster-up flag (Was=false) 2023-07-12 15:23:13,313 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-12 15:23:13,314 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase5.apache.org,39489,1689175392691 2023-07-12 15:23:13,316 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-12 15:23:13,316 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase5.apache.org,39489,1689175392691 2023-07-12 15:23:13,317 WARN [master/jenkins-hbase5:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.hbase-snapshot/.tmp 2023-07-12 15:23:13,318 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:13,318 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:13,318 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:13,318 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:13,318 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase5:0, corePoolSize=10, maxPoolSize=10 2023-07-12 15:23:13,318 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:13,318 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:23:13,318 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:13,320 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689175423320 2023-07-12 15:23:13,321 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-12 15:23:13,321 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-12 15:23:13,321 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-12 15:23:13,321 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-12 15:23:13,321 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-12 15:23:13,321 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-12 15:23:13,322 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:13,323 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-12 15:23:13,323 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-12 15:23:13,323 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-12 15:23:13,323 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-12 15:23:13,324 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-12 15:23:13,324 DEBUG [PEWorker-4] master.DeadServer(103): Processing jenkins-hbase5.apache.org,35897,1689175384731; numProcessing=1 2023-07-12 15:23:13,324 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase5.apache.org,38729,1689175384783; numProcessing=2 2023-07-12 15:23:13,324 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175393324,5,FailOnTimeoutGroup] 2023-07-12 15:23:13,324 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase5.apache.org,33173,1689175384809; numProcessing=3 2023-07-12 15:23:13,324 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase5.apache.org,42707,1689175384756; numProcessing=4 2023-07-12 15:23:13,325 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=15, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,33173,1689175384809, splitWal=true, meta=true 2023-07-12 15:23:13,325 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=16, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,38729,1689175384783, splitWal=true, meta=false 2023-07-12 15:23:13,325 INFO [PEWorker-4] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,35897,1689175384731, splitWal=true, meta=false 2023-07-12 15:23:13,325 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=17, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,42707,1689175384756, splitWal=true, meta=false 2023-07-12 15:23:13,326 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,33173,1689175384809, splitWal=true, meta=true, isMeta: true 2023-07-12 15:23:13,327 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175393324,5,FailOnTimeoutGroup] 2023-07-12 15:23:13,327 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:13,327 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-12 15:23:13,327 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:13,327 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:13,327 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689175393327, completionTime=-1 2023-07-12 15:23:13,327 WARN [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-12 15:23:13,327 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-12 15:23:13,329 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33173,1689175384809-splitting 2023-07-12 15:23:13,330 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33173,1689175384809-splitting dir is empty, no logs to split. 2023-07-12 15:23:13,330 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase5.apache.org,33173,1689175384809 WAL count=0, meta=true 2023-07-12 15:23:13,336 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33173,1689175384809-splitting dir is empty, no logs to split. 2023-07-12 15:23:13,336 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase5.apache.org,33173,1689175384809 WAL count=0, meta=true 2023-07-12 15:23:13,336 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase5.apache.org,33173,1689175384809 WAL splitting is done? wals=0, meta=true 2023-07-12 15:23:13,338 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-12 15:23:13,344 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-12 15:23:13,347 INFO [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(951): ClusterId : 5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:23:13,347 DEBUG [RS:0;jenkins-hbase5:40223] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 15:23:13,348 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-12 15:23:13,349 DEBUG [RS:0;jenkins-hbase5:40223] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 15:23:13,349 DEBUG [RS:0;jenkins-hbase5:40223] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 15:23:13,352 DEBUG [RS:0;jenkins-hbase5:40223] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 15:23:13,353 DEBUG [RS:0;jenkins-hbase5:40223] zookeeper.ReadOnlyZKClient(139): Connect 0x65d0dc9a to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:13,358 DEBUG [RS:0;jenkins-hbase5:40223] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b172e50, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:13,358 DEBUG [RS:0;jenkins-hbase5:40223] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6f28b7ea, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase5.apache.org/172.31.10.131:0 2023-07-12 15:23:13,368 DEBUG [RS:0;jenkins-hbase5:40223] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase5:40223 2023-07-12 15:23:13,368 INFO [RS:0;jenkins-hbase5:40223] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 15:23:13,368 INFO [RS:0;jenkins-hbase5:40223] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 15:23:13,368 DEBUG [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 15:23:13,368 INFO [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase5.apache.org,39489,1689175392691 with isa=jenkins-hbase5.apache.org/172.31.10.131:40223, startcode=1689175392718 2023-07-12 15:23:13,369 DEBUG [RS:0;jenkins-hbase5:40223] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 15:23:13,375 INFO [RS-EventLoopGroup-11-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:52581, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 15:23:13,376 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39489] master.ServerManager(394): Registering regionserver=jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:13,376 DEBUG [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143 2023-07-12 15:23:13,376 DEBUG [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:36223 2023-07-12 15:23:13,377 DEBUG [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 15:23:13,378 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=51ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-12 15:23:13,378 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:13,379 DEBUG [RS:0;jenkins-hbase5:40223] zookeeper.ZKUtil(162): regionserver:40223-0x101c5bd87ea001d, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:13,379 WARN [RS:0;jenkins-hbase5:40223] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:23:13,379 INFO [RS:0;jenkins-hbase5:40223] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:13,379 DEBUG [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(1948): logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:13,379 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase5.apache.org,40223,1689175392718] 2023-07-12 15:23:13,383 DEBUG [RS:0;jenkins-hbase5:40223] zookeeper.ZKUtil(162): regionserver:40223-0x101c5bd87ea001d, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:13,428 DEBUG [RS:0;jenkins-hbase5:40223] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-12 15:23:13,431 INFO [RS:0;jenkins-hbase5:40223] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 15:23:13,435 INFO [RS:0;jenkins-hbase5:40223] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 15:23:13,435 INFO [RS:0;jenkins-hbase5:40223] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 15:23:13,436 INFO [RS:0;jenkins-hbase5:40223] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:13,436 INFO [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 15:23:13,437 INFO [RS:0;jenkins-hbase5:40223] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:13,437 DEBUG [RS:0;jenkins-hbase5:40223] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:13,438 DEBUG [RS:0;jenkins-hbase5:40223] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:13,438 DEBUG [RS:0;jenkins-hbase5:40223] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:13,438 DEBUG [RS:0;jenkins-hbase5:40223] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:13,438 DEBUG [RS:0;jenkins-hbase5:40223] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:13,438 DEBUG [RS:0;jenkins-hbase5:40223] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase5:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:23:13,438 DEBUG [RS:0;jenkins-hbase5:40223] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:13,438 DEBUG [RS:0;jenkins-hbase5:40223] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:13,438 DEBUG [RS:0;jenkins-hbase5:40223] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:13,438 DEBUG [RS:0;jenkins-hbase5:40223] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:13,439 INFO [RS:0;jenkins-hbase5:40223] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:13,439 INFO [RS:0;jenkins-hbase5:40223] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:13,439 INFO [RS:0;jenkins-hbase5:40223] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:13,439 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-12 15:23:13,456 INFO [RS:0;jenkins-hbase5:40223] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 15:23:13,457 INFO [RS:0;jenkins-hbase5:40223] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,40223,1689175392718-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:13,475 INFO [RS:0;jenkins-hbase5:40223] regionserver.Replication(203): jenkins-hbase5.apache.org,40223,1689175392718 started 2023-07-12 15:23:13,475 INFO [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(1637): Serving as jenkins-hbase5.apache.org,40223,1689175392718, RpcServer on jenkins-hbase5.apache.org/172.31.10.131:40223, sessionid=0x101c5bd87ea001d 2023-07-12 15:23:13,475 DEBUG [RS:0;jenkins-hbase5:40223] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 15:23:13,475 DEBUG [RS:0;jenkins-hbase5:40223] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:13,475 DEBUG [RS:0;jenkins-hbase5:40223] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,40223,1689175392718' 2023-07-12 15:23:13,475 DEBUG [RS:0;jenkins-hbase5:40223] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-12 15:23:13,476 DEBUG [RS:0;jenkins-hbase5:40223] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-12 15:23:13,477 DEBUG [RS:0;jenkins-hbase5:40223] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 15:23:13,477 DEBUG [RS:0;jenkins-hbase5:40223] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 15:23:13,477 DEBUG [RS:0;jenkins-hbase5:40223] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:13,478 DEBUG [RS:0;jenkins-hbase5:40223] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,40223,1689175392718' 2023-07-12 15:23:13,478 DEBUG [RS:0;jenkins-hbase5:40223] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-12 15:23:13,478 DEBUG [RS:0;jenkins-hbase5:40223] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-12 15:23:13,479 DEBUG [RS:0;jenkins-hbase5:40223] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 15:23:13,479 INFO [RS:0;jenkins-hbase5:40223] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 15:23:13,479 INFO [RS:0;jenkins-hbase5:40223] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 15:23:13,498 DEBUG [jenkins-hbase5:39489] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-12 15:23:13,499 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase5.apache.org,40223,1689175392718, state=OPENING 2023-07-12 15:23:13,501 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-12 15:23:13,501 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-12 15:23:13,501 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase5.apache.org,40223,1689175392718}] 2023-07-12 15:23:13,581 INFO [RS:0;jenkins-hbase5:40223] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C40223%2C1689175392718, suffix=, logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,40223,1689175392718, archiveDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs, maxLogs=10 2023-07-12 15:23:13,603 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK] 2023-07-12 15:23:13,603 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK] 2023-07-12 15:23:13,603 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46687,DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4,DISK] 2023-07-12 15:23:13,606 INFO [RS:0;jenkins-hbase5:40223] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,40223,1689175392718/jenkins-hbase5.apache.org%2C40223%2C1689175392718.1689175393581 2023-07-12 15:23:13,606 DEBUG [RS:0;jenkins-hbase5:40223] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK], DatanodeInfoWithStorage[127.0.0.1:46687,DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4,DISK], DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK]] 2023-07-12 15:23:13,656 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:13,656 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:23:13,658 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:51196, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 15:23:13,661 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-12 15:23:13,661 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:13,663 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C40223%2C1689175392718.meta, suffix=.meta, logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,40223,1689175392718, archiveDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs, maxLogs=10 2023-07-12 15:23:13,679 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK] 2023-07-12 15:23:13,679 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46687,DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4,DISK] 2023-07-12 15:23:13,679 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK] 2023-07-12 15:23:13,682 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,40223,1689175392718/jenkins-hbase5.apache.org%2C40223%2C1689175392718.meta.1689175393664.meta 2023-07-12 15:23:13,682 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK], DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK], DatanodeInfoWithStorage[127.0.0.1:46687,DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4,DISK]] 2023-07-12 15:23:13,683 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:13,683 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-12 15:23:13,683 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-12 15:23:13,684 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-12 15:23:13,684 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-12 15:23:13,684 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:13,684 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-12 15:23:13,684 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-12 15:23:13,685 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-12 15:23:13,686 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info 2023-07-12 15:23:13,687 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info 2023-07-12 15:23:13,687 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-12 15:23:13,693 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info/f9013e31cdf14f9ca5ad1cff1f13869f 2023-07-12 15:23:13,694 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:13,694 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-12 15:23:13,695 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:23:13,695 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:23:13,695 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-12 15:23:13,696 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:13,696 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-12 15:23:13,697 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/table 2023-07-12 15:23:13,697 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/table 2023-07-12 15:23:13,697 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-12 15:23:13,703 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/table/351115f8a9734cc98bccaabf45019170 2023-07-12 15:23:13,703 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:13,704 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740 2023-07-12 15:23:13,705 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740 2023-07-12 15:23:13,707 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-12 15:23:13,708 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-12 15:23:13,709 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72242702, jitterRate=0.07650014758110046}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-12 15:23:13,709 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-12 15:23:13,710 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=20, masterSystemTime=1689175393656 2023-07-12 15:23:13,714 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-12 15:23:13,715 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-12 15:23:13,715 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase5.apache.org,40223,1689175392718, state=OPEN 2023-07-12 15:23:13,717 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-12 15:23:13,718 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-12 15:23:13,719 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-12 15:23:13,719 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase5.apache.org,40223,1689175392718 in 217 msec 2023-07-12 15:23:13,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=15 2023-07-12 15:23:13,722 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 383 msec 2023-07-12 15:23:14,880 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1553ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-07-12 15:23:16,383 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3056ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-12 15:23:17,835 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4508ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-12 15:23:17,835 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:23:17,839 DEBUG [hconnection-0x7b3290a5-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:17,840 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:55052, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:17,849 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=e7e73f29c14aa26ac2fc8049609eeb2d, regionState=OPEN, lastHost=jenkins-hbase5.apache.org,42707,1689175384756, regionLocation=jenkins-hbase5.apache.org,42707,1689175384756, openSeqNum=2 2023-07-12 15:23:17,849 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=19659d0923548261983bde115dd510a9, regionState=OPEN, lastHost=jenkins-hbase5.apache.org,33173,1689175384809, regionLocation=jenkins-hbase5.apache.org,33173,1689175384809, openSeqNum=2 2023-07-12 15:23:17,850 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=144fbe8ee3e556340eac4ab1f11a0604, regionState=OPEN, lastHost=jenkins-hbase5.apache.org,42707,1689175384756, regionLocation=jenkins-hbase5.apache.org,42707,1689175384756, openSeqNum=2 2023-07-12 15:23:17,850 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-12 15:23:17,850 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689175457850 2023-07-12 15:23:17,850 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689175517850 2023-07-12 15:23:17,850 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 14 msec 2023-07-12 15:23:17,874 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,39489,1689175392691-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:17,875 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,39489,1689175392691-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:17,875 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase5.apache.org,35897,1689175384731 had 0 regions 2023-07-12 15:23:17,875 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,39489,1689175392691-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:17,875 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase5.apache.org,33173,1689175384809 had 2 regions 2023-07-12 15:23:17,875 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase5.apache.org,42707,1689175384756 had 2 regions 2023-07-12 15:23:17,875 INFO [PEWorker-1] procedure.ServerCrashProcedure(199): jenkins-hbase5.apache.org,38729,1689175384783 had 0 regions 2023-07-12 15:23:17,875 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase5:39489, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:17,875 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:17,876 WARN [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. is NOT online; state={e7e73f29c14aa26ac2fc8049609eeb2d state=OPEN, ts=1689175397849, server=jenkins-hbase5.apache.org,42707,1689175384756}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-12 15:23:17,877 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,35897,1689175384731, splitWal=true, meta=false, isMeta: false 2023-07-12 15:23:17,877 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=17, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,42707,1689175384756, splitWal=true, meta=false, isMeta: false 2023-07-12 15:23:17,877 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,38729,1689175384783, splitWal=true, meta=false, isMeta: false 2023-07-12 15:23:17,877 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,33173,1689175384809, splitWal=true, meta=true, isMeta: false 2023-07-12 15:23:17,879 DEBUG [PEWorker-3] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,35897,1689175384731-splitting 2023-07-12 15:23:17,881 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,35897,1689175384731-splitting dir is empty, no logs to split. 2023-07-12 15:23:17,881 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase5.apache.org,35897,1689175384731 WAL count=0, meta=false 2023-07-12 15:23:17,881 DEBUG [PEWorker-4] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,42707,1689175384756-splitting 2023-07-12 15:23:17,882 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,42707,1689175384756-splitting dir is empty, no logs to split. 2023-07-12 15:23:17,882 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase5.apache.org,42707,1689175384756 WAL count=0, meta=false 2023-07-12 15:23:17,883 WARN [master/jenkins-hbase5:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase5.apache.org,42707,1689175384756/hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d., unknown_server=jenkins-hbase5.apache.org,33173,1689175384809/t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9., unknown_server=jenkins-hbase5.apache.org,42707,1689175384756/t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:17,883 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,38729,1689175384783-splitting 2023-07-12 15:23:17,884 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,38729,1689175384783-splitting dir is empty, no logs to split. 2023-07-12 15:23:17,884 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase5.apache.org,38729,1689175384783 WAL count=0, meta=false 2023-07-12 15:23:17,886 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33173,1689175384809-splitting dir is empty, no logs to split. 2023-07-12 15:23:17,886 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase5.apache.org,33173,1689175384809 WAL count=0, meta=false 2023-07-12 15:23:17,887 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,35897,1689175384731-splitting dir is empty, no logs to split. 2023-07-12 15:23:17,887 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase5.apache.org,35897,1689175384731 WAL count=0, meta=false 2023-07-12 15:23:17,887 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase5.apache.org,35897,1689175384731 WAL splitting is done? wals=0, meta=false 2023-07-12 15:23:17,888 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,42707,1689175384756-splitting dir is empty, no logs to split. 2023-07-12 15:23:17,888 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase5.apache.org,42707,1689175384756 WAL count=0, meta=false 2023-07-12 15:23:17,888 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase5.apache.org,42707,1689175384756 WAL splitting is done? wals=0, meta=false 2023-07-12 15:23:17,890 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,38729,1689175384783-splitting dir is empty, no logs to split. 2023-07-12 15:23:17,890 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase5.apache.org,38729,1689175384783 WAL count=0, meta=false 2023-07-12 15:23:17,890 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase5.apache.org,38729,1689175384783 WAL splitting is done? wals=0, meta=false 2023-07-12 15:23:17,890 INFO [PEWorker-3] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase5.apache.org,35897,1689175384731 failed, ignore...File hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,35897,1689175384731-splitting does not exist. 2023-07-12 15:23:17,890 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33173,1689175384809-splitting dir is empty, no logs to split. 2023-07-12 15:23:17,890 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase5.apache.org,33173,1689175384809 WAL count=0, meta=false 2023-07-12 15:23:17,891 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase5.apache.org,33173,1689175384809 WAL splitting is done? wals=0, meta=false 2023-07-12 15:23:17,892 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase5.apache.org,35897,1689175384731 after splitting done 2023-07-12 15:23:17,892 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase5.apache.org,35897,1689175384731 from processing; numProcessing=3 2023-07-12 15:23:17,893 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=19659d0923548261983bde115dd510a9, ASSIGN}] 2023-07-12 15:23:17,893 INFO [PEWorker-1] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase5.apache.org,38729,1689175384783 failed, ignore...File hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,38729,1689175384783-splitting does not exist. 2023-07-12 15:23:17,893 INFO [PEWorker-4] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase5.apache.org,42707,1689175384756 failed, ignore...File hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,42707,1689175384756-splitting does not exist. 2023-07-12 15:23:17,894 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=19659d0923548261983bde115dd510a9, ASSIGN 2023-07-12 15:23:17,895 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase5.apache.org,38729,1689175384783 after splitting done 2023-07-12 15:23:17,895 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e7e73f29c14aa26ac2fc8049609eeb2d, ASSIGN}, {pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=144fbe8ee3e556340eac4ab1f11a0604, ASSIGN}] 2023-07-12 15:23:17,895 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase5.apache.org,38729,1689175384783 from processing; numProcessing=2 2023-07-12 15:23:17,895 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=19659d0923548261983bde115dd510a9, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-12 15:23:17,895 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase5.apache.org,35897,1689175384731, splitWal=true, meta=false in 4.6000 sec 2023-07-12 15:23:17,896 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e7e73f29c14aa26ac2fc8049609eeb2d, ASSIGN 2023-07-12 15:23:17,896 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=144fbe8ee3e556340eac4ab1f11a0604, ASSIGN 2023-07-12 15:23:17,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase5.apache.org,38729,1689175384783, splitWal=true, meta=false in 4.6070 sec 2023-07-12 15:23:17,897 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=e7e73f29c14aa26ac2fc8049609eeb2d, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-12 15:23:17,897 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=144fbe8ee3e556340eac4ab1f11a0604, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-12 15:23:17,897 DEBUG [jenkins-hbase5:39489] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-12 15:23:17,899 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=19659d0923548261983bde115dd510a9, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:17,900 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175397899"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175397899"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175397899"}]},"ts":"1689175397899"} 2023-07-12 15:23:17,900 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=e7e73f29c14aa26ac2fc8049609eeb2d, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:17,900 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689175397900"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175397900"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175397900"}]},"ts":"1689175397900"} 2023-07-12 15:23:17,902 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=21, state=RUNNABLE; OpenRegionProcedure 19659d0923548261983bde115dd510a9, server=jenkins-hbase5.apache.org,40223,1689175392718}] 2023-07-12 15:23:17,904 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure e7e73f29c14aa26ac2fc8049609eeb2d, server=jenkins-hbase5.apache.org,40223,1689175392718}] 2023-07-12 15:23:18,051 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=144fbe8ee3e556340eac4ab1f11a0604, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:18,051 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175398051"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175398051"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175398051"}]},"ts":"1689175398051"} 2023-07-12 15:23:18,053 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=23, state=RUNNABLE; OpenRegionProcedure 144fbe8ee3e556340eac4ab1f11a0604, server=jenkins-hbase5.apache.org,40223,1689175392718}] 2023-07-12 15:23:18,058 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:18,058 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e7e73f29c14aa26ac2fc8049609eeb2d, NAME => 'hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:18,059 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:18,059 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:18,059 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:18,059 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:18,060 INFO [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:18,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:18,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 19659d0923548261983bde115dd510a9, NAME => 't1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:18,061 DEBUG [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/info 2023-07-12 15:23:18,061 DEBUG [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/info 2023-07-12 15:23:18,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 19659d0923548261983bde115dd510a9 2023-07-12 15:23:18,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:18,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 19659d0923548261983bde115dd510a9 2023-07-12 15:23:18,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 19659d0923548261983bde115dd510a9 2023-07-12 15:23:18,062 INFO [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e7e73f29c14aa26ac2fc8049609eeb2d columnFamilyName info 2023-07-12 15:23:18,063 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 19659d0923548261983bde115dd510a9 2023-07-12 15:23:18,064 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1 2023-07-12 15:23:18,064 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1 2023-07-12 15:23:18,064 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 19659d0923548261983bde115dd510a9 columnFamilyName cf1 2023-07-12 15:23:18,068 DEBUG [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/info/98bda8b543444aaf919145966712fca0 2023-07-12 15:23:18,070 INFO [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] regionserver.HStore(310): Store=e7e73f29c14aa26ac2fc8049609eeb2d/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:18,071 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:18,072 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:18,073 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1/82858d2ae5f44600b94c571e38fb835c 2023-07-12 15:23:18,073 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(310): Store=19659d0923548261983bde115dd510a9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:18,073 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 19659d0923548261983bde115dd510a9 2023-07-12 15:23:18,074 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/norep 2023-07-12 15:23:18,074 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/norep 2023-07-12 15:23:18,074 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 19659d0923548261983bde115dd510a9 columnFamilyName norep 2023-07-12 15:23:18,075 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(310): Store=19659d0923548261983bde115dd510a9/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:18,075 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:18,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9 2023-07-12 15:23:18,076 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened e7e73f29c14aa26ac2fc8049609eeb2d; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67825456, jitterRate=0.01067805290222168}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:23:18,076 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for e7e73f29c14aa26ac2fc8049609eeb2d: 2023-07-12 15:23:18,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9 2023-07-12 15:23:18,077 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d., pid=25, masterSystemTime=1689175398054 2023-07-12 15:23:18,079 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:18,079 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:18,080 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=e7e73f29c14aa26ac2fc8049609eeb2d, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:18,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:23:18,080 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689175398080"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175398080"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175398080"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175398080"}]},"ts":"1689175398080"} 2023-07-12 15:23:18,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 19659d0923548261983bde115dd510a9 2023-07-12 15:23:18,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 19659d0923548261983bde115dd510a9; next sequenceid=11; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69803740, jitterRate=0.040156781673431396}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:23:18,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 19659d0923548261983bde115dd510a9: 2023-07-12 15:23:18,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9., pid=24, masterSystemTime=1689175398054 2023-07-12 15:23:18,083 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-07-12 15:23:18,084 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure e7e73f29c14aa26ac2fc8049609eeb2d, server=jenkins-hbase5.apache.org,40223,1689175392718 in 178 msec 2023-07-12 15:23:18,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:18,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:18,085 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=e7e73f29c14aa26ac2fc8049609eeb2d, ASSIGN in 189 msec 2023-07-12 15:23:18,085 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=19659d0923548261983bde115dd510a9, regionState=OPEN, openSeqNum=11, regionLocation=jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:18,086 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175398085"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175398085"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175398085"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175398085"}]},"ts":"1689175398085"} 2023-07-12 15:23:18,088 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=21 2023-07-12 15:23:18,089 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=21, state=SUCCESS; OpenRegionProcedure 19659d0923548261983bde115dd510a9, server=jenkins-hbase5.apache.org,40223,1689175392718 in 185 msec 2023-07-12 15:23:18,090 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=15 2023-07-12 15:23:18,090 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase5.apache.org,33173,1689175384809 after splitting done 2023-07-12 15:23:18,090 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=19659d0923548261983bde115dd510a9, ASSIGN in 196 msec 2023-07-12 15:23:18,090 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase5.apache.org,33173,1689175384809 from processing; numProcessing=1 2023-07-12 15:23:18,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase5.apache.org,33173,1689175384809, splitWal=true, meta=true in 4.8110 sec 2023-07-12 15:23:18,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:18,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 144fbe8ee3e556340eac4ab1f11a0604, NAME => 't2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:18,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:18,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:18,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:18,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:18,211 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:18,212 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1 2023-07-12 15:23:18,212 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1 2023-07-12 15:23:18,212 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 144fbe8ee3e556340eac4ab1f11a0604 columnFamilyName cf1 2023-07-12 15:23:18,218 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1/90e6bd84cbb748218903eb87a1f792f7 2023-07-12 15:23:18,218 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(310): Store=144fbe8ee3e556340eac4ab1f11a0604/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:18,219 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:18,220 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/norep 2023-07-12 15:23:18,220 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/norep 2023-07-12 15:23:18,220 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 144fbe8ee3e556340eac4ab1f11a0604 columnFamilyName norep 2023-07-12 15:23:18,221 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(310): Store=144fbe8ee3e556340eac4ab1f11a0604/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:18,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:18,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:18,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:23:18,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:18,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 144fbe8ee3e556340eac4ab1f11a0604; next sequenceid=11; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63274080, jitterRate=-0.05714273452758789}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:23:18,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 144fbe8ee3e556340eac4ab1f11a0604: 2023-07-12 15:23:18,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604., pid=26, masterSystemTime=1689175398205 2023-07-12 15:23:18,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:18,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:18,230 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=144fbe8ee3e556340eac4ab1f11a0604, regionState=OPEN, openSeqNum=11, regionLocation=jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:18,230 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175398230"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175398230"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175398230"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175398230"}]},"ts":"1689175398230"} 2023-07-12 15:23:18,233 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=23 2023-07-12 15:23:18,233 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=23, state=SUCCESS; OpenRegionProcedure 144fbe8ee3e556340eac4ab1f11a0604, server=jenkins-hbase5.apache.org,40223,1689175392718 in 178 msec 2023-07-12 15:23:18,235 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=17 2023-07-12 15:23:18,235 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase5.apache.org,42707,1689175384756 after splitting done 2023-07-12 15:23:18,235 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=144fbe8ee3e556340eac4ab1f11a0604, ASSIGN in 338 msec 2023-07-12 15:23:18,235 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase5.apache.org,42707,1689175384756 from processing; numProcessing=0 2023-07-12 15:23:18,236 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase5.apache.org,42707,1689175384756, splitWal=true, meta=false in 4.9450 sec 2023-07-12 15:23:18,474 INFO [jenkins-hbase5:40223Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:23:18,877 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-12 15:23:18,890 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-12 15:23:18,893 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-12 15:23:18,893 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 6.149sec 2023-07-12 15:23:18,893 INFO [master/jenkins-hbase5:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-12 15:23:18,893 INFO [master/jenkins-hbase5:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-12 15:23:18,893 INFO [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-12 15:23:18,894 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,39489,1689175392691-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-12 15:23:18,894 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,39489,1689175392691-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-12 15:23:18,897 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-12 15:23:18,948 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(139): Connect 0x4013ba43 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:18,953 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5dcc4257, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:18,954 DEBUG [hconnection-0x2d47f211-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:18,956 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:55062, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:18,959 INFO [Listener at localhost/42735] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-12 15:23:18,959 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4013ba43 to 127.0.0.1:64130 2023-07-12 15:23:18,959 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:18,960 INFO [Listener at localhost/42735] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase5.apache.org:39489 after: jenkins-hbase5.apache.org:39489 2023-07-12 15:23:18,961 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(139): Connect 0x1905cfa5 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:18,965 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@b4061ee, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:19,187 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-12 15:23:19,430 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-12 15:23:19,431 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-12 15:23:19,431 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-12 15:23:19,432 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-12 15:23:19,466 DEBUG [hconnection-0x349d418f-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:19,468 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:55068, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:19,491 INFO [Listener at localhost/42735] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689175399491 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:19,494 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): syncupReplication16891753994910x0, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:19,495 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689175399491-0x101c5bd87ea0022 connected Start Replication Server start 2023-07-12 15:23:19,498 INFO [Listener at localhost/42735] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:19,510 INFO [Listener at localhost/42735] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-12 15:23:19,511 DEBUG [Listener at localhost/42735] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-12 15:23:19,523 INFO [Listener at localhost/42735] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-12 15:23:19,524 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase5.apache.org,36017,1689175380873, jenkins-hbase5.apache.org,35567,1689175380829] other RSs: [] 2023-07-12 15:23:19,543 DEBUG [Listener at localhost/42735.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x407c2a80 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:19,549 DEBUG [Listener at localhost/42735.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@550c031e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:19,549 INFO [Listener at localhost/42735.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:19,552 DEBUG [Listener at localhost/42735.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:19,553 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101c5bd87ea0024 connected 2023-07-12 15:23:19,554 INFO [Listener at localhost/42735.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=098cb589-347a-4654-bf7c-f2a1c06d1dde to cluster=5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:23:23,231 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase5.apache.org,36017,1689175380873/1's WALs to 1689175399498.syncuptool.replication.org,1234,1 2023-07-12 15:23:23,237 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase5.apache.org%2C36017%2C1689175380873.1689175382331 with data PBUF\x08\x84\xFF\x01 2023-07-12 15:23:23,239 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase5.apache.org,36017,1689175380873/1's WALs to 1689175399498.syncuptool.replication.org,1234,1 2023-07-12 15:23:23,474 INFO [jenkins-hbase5:40223Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:23:25,241 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase5.apache.org,36017,1689175380873] 2023-07-12 15:23:25,241 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase5.apache.org,36017,1689175380873] 2023-07-12 15:23:25,255 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase5.apache.org,36017,1689175380873] 2023-07-12 15:23:25,255 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase5.apache.org,36017,1689175380873, ReplicationSource: 1, currentBandwidth=0 2023-07-12 15:23:25,255 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:25,281 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873] zookeeper.ReadOnlyZKClient(139): Connect 0x04eafa70 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:25,286 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@1f0b849b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:25,286 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:25,290 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:25,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101c5bd87ea0026 connected 2023-07-12 15:23:25,291 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase5.apache.org,36017,1689175380873 (queues=1) is replicating from cluster=098cb589-347a-4654-bf7c-f2a1c06d1dde to cluster=5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:23:25,291 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase5.apache.org%2C36017%2C1689175380873 2023-07-12 15:23:25,295 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase5.apache.org,36017,1689175380873, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-12 15:23:25,296 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1-jenkins-hbase5.apache.org,36017,1689175380873] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase5.apache.org%2C36017%2C1689175380873 2023-07-12 15:23:25,305 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1-jenkins-hbase5.apache.org,36017,1689175380873] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs/jenkins-hbase5.apache.org%2C36017%2C1689175380873.1689175382331 2023-07-12 15:23:25,305 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1-jenkins-hbase5.apache.org,36017,1689175380873] regionserver.ReplicationSourceWALReader(162): Read 2 WAL entries eligible for replication 2023-07-12 15:23:25,305 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C36017%2C1689175380873,1-jenkins-hbase5.apache.org,36017,1689175380873] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-12 15:23:25,306 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1-jenkins-hbase5.apache.org,36017,1689175380873] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase5.apache.org,36017,1689175380873 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/e2a1d8488c64dcf2f22986998cf96932/105=[#edits: 50 = ],4120}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs/jenkins-hbase5.apache.org%2C36017%2C1689175380873.1689175382331, lastWalPosition=44348, nbRowKeys=150, nbHFiles=0, heapSize=12240, lastSeqIds={}, endOfFile=true,usedBufferSize=12240] 2023-07-12 15:23:25,308 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:23:25,310 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:55070, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 15:23:25,312 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40223] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-12 15:23:25,312 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40223] zookeeper.ReadOnlyZKClient(139): Connect 0x05555d87 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:25,319 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40223] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@b92a35d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:25,321 DEBUG [hconnection-0xfad36f7-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:25,322 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:55074, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=ClientService 2023-07-12 15:23:25,338 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40223] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-12 15:23:25,338 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1-jenkins-hbase5.apache.org,36017,1689175380873] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase5.apache.org,36017,1689175380873] 2023-07-12 15:23:25,340 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1-jenkins-hbase5.apache.org,36017,1689175380873] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase5.apache.org%2C36017%2C1689175380873.1689175382331] 2023-07-12 15:23:25,341 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1-jenkins-hbase5.apache.org,36017,1689175380873] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase5.apache.org,36017,1689175380873 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-12 15:23:25,342 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1-jenkins-hbase5.apache.org,36017,1689175380873] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase5.apache.org%2C36017%2C1689175380873 of peer 1-jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:25,442 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1-jenkins-hbase5.apache.org,36017,1689175380873] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:25,444 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689175399491-0x101c5bd87ea0022, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689175399498.syncuptool.replication.org,1234,1/1-jenkins-hbase5.apache.org,36017,1689175380873 2023-07-12 15:23:25,444 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873.replicationSource.shipperjenkins-hbase5.apache.org%2C36017%2C1689175380873,1-jenkins-hbase5.apache.org,36017,1689175380873] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase5.apache.org,36017,1689175380873 with the following stats: Total replicated edits: 2, current progress: 2023-07-12 15:23:27,755 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase5.apache.org,35567,1689175380829/1's WALs to 1689175399498.syncuptool.replication.org,1234,1 2023-07-12 15:23:27,758 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase5.apache.org%2C35567%2C1689175380829.1689175382331 with data PBUF\x08\xCD\x06 2023-07-12 15:23:27,760 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase5.apache.org,35567,1689175380829/1's WALs to 1689175399498.syncuptool.replication.org,1234,1 2023-07-12 15:23:28,474 INFO [jenkins-hbase5:40223Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:23:28,474 INFO [jenkins-hbase5:40223Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-12 15:23:29,762 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase5.apache.org,35567,1689175380829] 2023-07-12 15:23:29,762 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase5.apache.org,35567,1689175380829] 2023-07-12 15:23:29,774 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase5.apache.org,35567,1689175380829] 2023-07-12 15:23:29,774 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase5.apache.org,35567,1689175380829, ReplicationSource: 1, currentBandwidth=0 2023-07-12 15:23:29,774 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:29,802 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829] zookeeper.ReadOnlyZKClient(139): Connect 0x7b98a54a to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:29,807 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@265d1845, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:29,808 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:29,811 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:29,812 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase5.apache.org,35567,1689175380829 (queues=1) is replicating from cluster=098cb589-347a-4654-bf7c-f2a1c06d1dde to cluster=5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:23:29,812 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase5.apache.org%2C35567%2C1689175380829 2023-07-12 15:23:29,815 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101c5bd87ea0029 connected 2023-07-12 15:23:29,816 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase5.apache.org,35567,1689175380829, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-12 15:23:29,820 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829.replicationSource.shipperjenkins-hbase5.apache.org%2C35567%2C1689175380829,1-jenkins-hbase5.apache.org,35567,1689175380829] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase5.apache.org%2C35567%2C1689175380829 2023-07-12 15:23:29,828 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1-jenkins-hbase5.apache.org,35567,1689175380829] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs/jenkins-hbase5.apache.org%2C35567%2C1689175380829.1689175382331 2023-07-12 15:23:29,828 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1-jenkins-hbase5.apache.org,35567,1689175380829] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-12 15:23:29,828 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C35567%2C1689175380829,1-jenkins-hbase5.apache.org,35567,1689175380829] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-12 15:23:29,828 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829.replicationSource.shipperjenkins-hbase5.apache.org%2C35567%2C1689175380829,1-jenkins-hbase5.apache.org,35567,1689175380829] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase5.apache.org,35567,1689175380829 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs/jenkins-hbase5.apache.org%2C35567%2C1689175380829.1689175382331, lastWalPosition=1430, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-07-12 15:23:29,829 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829.replicationSource.shipperjenkins-hbase5.apache.org%2C35567%2C1689175380829,1-jenkins-hbase5.apache.org,35567,1689175380829] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase5.apache.org,35567,1689175380829] 2023-07-12 15:23:29,832 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829.replicationSource.shipperjenkins-hbase5.apache.org%2C35567%2C1689175380829,1-jenkins-hbase5.apache.org,35567,1689175380829] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase5.apache.org%2C35567%2C1689175380829.1689175382331] 2023-07-12 15:23:29,833 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829.replicationSource.shipperjenkins-hbase5.apache.org%2C35567%2C1689175380829,1-jenkins-hbase5.apache.org,35567,1689175380829] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase5.apache.org,35567,1689175380829 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-12 15:23:29,833 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829.replicationSource.shipperjenkins-hbase5.apache.org%2C35567%2C1689175380829,1-jenkins-hbase5.apache.org,35567,1689175380829] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase5.apache.org%2C35567%2C1689175380829 of peer 1-jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:29,933 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829.replicationSource.shipperjenkins-hbase5.apache.org%2C35567%2C1689175380829,1-jenkins-hbase5.apache.org,35567,1689175380829] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:29,936 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689175399491-0x101c5bd87ea0022, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689175399498.syncuptool.replication.org,1234,1/1-jenkins-hbase5.apache.org,35567,1689175380829 2023-07-12 15:23:29,936 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829.replicationSource.shipperjenkins-hbase5.apache.org%2C35567%2C1689175380829,1-jenkins-hbase5.apache.org,35567,1689175380829] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase5.apache.org,35567,1689175380829 with the following stats: Total replicated edits: 0, current progress: 2023-07-12 15:23:31,007 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-12 15:23:33,474 INFO [jenkins-hbase5:40223Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:23:33,474 INFO [jenkins-hbase5:40223Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-12 15:23:38,474 INFO [jenkins-hbase5:40223Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:23:38,474 INFO [jenkins-hbase5:40223Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-12 15:23:39,525 INFO [Listener at localhost/42735] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-12 15:23:39,629 DEBUG [Listener at localhost/42735.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101c5bd87ea0024, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:39,629 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x407c2a80 to 127.0.0.1:64130 2023-07-12 15:23:39,630 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:39,832 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689175399491-0x101c5bd87ea0022, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:39,832 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689175399491-0x101c5bd87ea0022, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:39,846 INFO [Listener at localhost/42735] replication.TestReplicationSyncUpTool(203): SyncUpAfterDelete succeeded at retry = 0 2023-07-12 15:23:39,846 DEBUG [Listener at localhost/42735] replication.TestReplicationSyncUpTool(215): mimicSyncUpAfterPut 2023-07-12 15:23:39,848 INFO [Listener at localhost/42735] client.ConnectionUtils(127): master/jenkins-hbase5:0 server-side Connection retries=45 2023-07-12 15:23:39,848 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:39,848 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:39,848 INFO [Listener at localhost/42735] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:23:39,849 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:39,849 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:23:39,849 INFO [Listener at localhost/42735] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:23:39,850 INFO [Listener at localhost/42735] ipc.NettyRpcServer(120): Bind to /172.31.10.131:33093 2023-07-12 15:23:39,850 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:39,852 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:39,853 INFO [Listener at localhost/42735] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33093 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:39,855 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:330930x0, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:39,857 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:33093-0x101c5bd87ea002a connected 2023-07-12 15:23:39,859 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-12 15:23:39,860 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-12 15:23:39,860 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-12 15:23:39,860 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33093 2023-07-12 15:23:39,863 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33093 2023-07-12 15:23:39,866 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33093 2023-07-12 15:23:39,866 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33093 2023-07-12 15:23:39,869 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33093 2023-07-12 15:23:39,869 INFO [Listener at localhost/42735] master.HMaster(444): hbase.rootdir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4, hbase.cluster.distributed=false 2023-07-12 15:23:39,870 DEBUG [pool-541-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: INIT 2023-07-12 15:23:39,881 INFO [Listener at localhost/42735] client.ConnectionUtils(127): regionserver/jenkins-hbase5:0 server-side Connection retries=45 2023-07-12 15:23:39,881 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:39,881 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:39,881 INFO [Listener at localhost/42735] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:23:39,881 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:39,881 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:23:39,881 INFO [Listener at localhost/42735] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:23:39,882 INFO [Listener at localhost/42735] ipc.NettyRpcServer(120): Bind to /172.31.10.131:43385 2023-07-12 15:23:39,882 INFO [Listener at localhost/42735] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 15:23:39,884 DEBUG [Listener at localhost/42735] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 15:23:39,884 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:39,885 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:39,886 INFO [Listener at localhost/42735] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43385 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:39,894 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:433850x0, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:39,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:43385-0x101c5bd87ea002b connected 2023-07-12 15:23:39,896 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:43385-0x101c5bd87ea002b, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-12 15:23:39,897 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:43385-0x101c5bd87ea002b, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-12 15:23:39,897 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:43385-0x101c5bd87ea002b, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-12 15:23:39,900 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43385 2023-07-12 15:23:39,901 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43385 2023-07-12 15:23:39,903 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43385 2023-07-12 15:23:39,903 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43385 2023-07-12 15:23:39,903 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43385 2023-07-12 15:23:39,905 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase5.apache.org,33093,1689175419847 2023-07-12 15:23:39,907 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-12 15:23:39,907 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase5.apache.org,33093,1689175419847 2023-07-12 15:23:39,908 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-12 15:23:39,908 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:43385-0x101c5bd87ea002b, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-12 15:23:39,908 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:39,909 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-12 15:23:39,911 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-12 15:23:39,911 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase5.apache.org,33093,1689175419847 from backup master directory 2023-07-12 15:23:39,912 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase5.apache.org,33093,1689175419847 2023-07-12 15:23:39,912 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-12 15:23:39,912 WARN [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:23:39,912 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase5.apache.org,33093,1689175419847 2023-07-12 15:23:39,932 INFO [master/jenkins-hbase5:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:39,966 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x1a289340 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:39,970 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4462ed15, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:39,970 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-12 15:23:39,971 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-12 15:23:39,971 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:39,978 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/WALs/jenkins-hbase5.apache.org,45903,1689175379564 to hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/WALs/jenkins-hbase5.apache.org,45903,1689175379564-dead as it is dead 2023-07-12 15:23:39,978 INFO [master/jenkins-hbase5:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/WALs/jenkins-hbase5.apache.org,45903,1689175379564-dead/jenkins-hbase5.apache.org%2C45903%2C1689175379564.1689175381318 2023-07-12 15:23:39,979 INFO [master/jenkins-hbase5:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/WALs/jenkins-hbase5.apache.org,45903,1689175379564-dead/jenkins-hbase5.apache.org%2C45903%2C1689175379564.1689175381318 after 1ms 2023-07-12 15:23:39,980 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/WALs/jenkins-hbase5.apache.org,45903,1689175379564-dead/jenkins-hbase5.apache.org%2C45903%2C1689175379564.1689175381318 to hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase5.apache.org%2C45903%2C1689175379564.1689175381318 2023-07-12 15:23:39,980 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/WALs/jenkins-hbase5.apache.org,45903,1689175379564-dead 2023-07-12 15:23:39,980 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/WALs/jenkins-hbase5.apache.org,33093,1689175419847 2023-07-12 15:23:39,983 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C33093%2C1689175419847, suffix=, logDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/WALs/jenkins-hbase5.apache.org,33093,1689175419847, archiveDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/oldWALs, maxLogs=10 2023-07-12 15:23:39,998 DEBUG [RS-EventLoopGroup-15-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38179,DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770,DISK] 2023-07-12 15:23:39,998 DEBUG [RS-EventLoopGroup-15-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34197,DS-999b06e9-279a-47df-944d-eabe177df1c6,DISK] 2023-07-12 15:23:40,001 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/WALs/jenkins-hbase5.apache.org,33093,1689175419847/jenkins-hbase5.apache.org%2C33093%2C1689175419847.1689175419983 2023-07-12 15:23:40,003 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38179,DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770,DISK], DatanodeInfoWithStorage[127.0.0.1:34197,DS-999b06e9-279a-47df-944d-eabe177df1c6,DISK]] 2023-07-12 15:23:40,003 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:40,003 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:40,003 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:40,003 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:40,007 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:40,008 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-12 15:23:40,009 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-12 15:23:40,015 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e3ba367d06984e51a93df5b25e050654 2023-07-12 15:23:40,015 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:40,016 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-12 15:23:40,016 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase5.apache.org%2C45903%2C1689175379564.1689175381318 2023-07-12 15:23:40,021 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 155, firstSequenceIdInLog=3, maxSequenceIdInLog=138, path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase5.apache.org%2C45903%2C1689175379564.1689175381318 2023-07-12 15:23:40,022 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase5.apache.org%2C45903%2C1689175379564.1689175381318 2023-07-12 15:23:40,025 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:40,027 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/138.seqid, newMaxSeqId=138, maxSeqId=1 2023-07-12 15:23:40,028 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=139; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59491901, jitterRate=-0.11350159347057343}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:23:40,028 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:23:40,028 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-12 15:23:40,030 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-12 15:23:40,030 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-12 15:23:40,030 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:23:40,031 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:23:40,035 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:23:40,036 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:23:40,036 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:23:40,037 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:23:40,037 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-12 15:23:40,037 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-12 15:23:40,038 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-12 15:23:40,038 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 15:23:40,038 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-12 15:23:40,039 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:23:40,040 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase5.apache.org,35567,1689175380829, table=hbase:meta, region=1588230740 2023-07-12 15:23:40,041 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 2 possibly 'live' servers, and 0 'splitting'. 2023-07-12 15:23:40,043 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase5.apache.org,35567,1689175380829 already deleted, retry=false 2023-07-12 15:23:40,043 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase5.apache.org,35567,1689175380829 on jenkins-hbase5.apache.org,33093,1689175419847 2023-07-12 15:23:40,044 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase5.apache.org,35567,1689175380829, splitWal=true, meta=true 2023-07-12 15:23:40,044 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase5.apache.org,35567,1689175380829 (carryingMeta=true) jenkins-hbase5.apache.org,35567,1689175380829/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@12fd2e86[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-12 15:23:40,046 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase5.apache.org,36017,1689175380873 already deleted, retry=false 2023-07-12 15:23:40,046 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase5.apache.org,36017,1689175380873 on jenkins-hbase5.apache.org,33093,1689175419847 2023-07-12 15:23:40,047 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase5.apache.org,36017,1689175380873, splitWal=true, meta=false 2023-07-12 15:23:40,047 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=19 for jenkins-hbase5.apache.org,36017,1689175380873 (carryingMeta=false) jenkins-hbase5.apache.org,36017,1689175380873/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@64f2c356[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-12 15:23:40,063 INFO [master/jenkins-hbase5:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-12 15:23:40,063 INFO [master/jenkins-hbase5:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-12 15:23:40,063 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-12 15:23:40,064 INFO [master/jenkins-hbase5:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-12 15:23:40,064 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-12 15:23:40,065 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-12 15:23:40,065 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-12 15:23:40,069 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-12 15:23:40,070 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-12 15:23:40,070 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:43385-0x101c5bd87ea002b, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-12 15:23:40,070 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:40,071 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase5.apache.org,33093,1689175419847, sessionid=0x101c5bd87ea002a, setting cluster-up flag (Was=false) 2023-07-12 15:23:40,077 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-12 15:23:40,077 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase5.apache.org,33093,1689175419847 2023-07-12 15:23:40,080 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-12 15:23:40,081 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase5.apache.org,33093,1689175419847 2023-07-12 15:23:40,081 WARN [master/jenkins-hbase5:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/.hbase-snapshot/.tmp 2023-07-12 15:23:40,083 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:40,083 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:40,083 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:40,083 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:40,083 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase5:0, corePoolSize=10, maxPoolSize=10 2023-07-12 15:23:40,083 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:40,083 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:23:40,083 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:40,084 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689175450084 2023-07-12 15:23:40,085 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-12 15:23:40,085 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-12 15:23:40,085 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-12 15:23:40,085 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-12 15:23:40,085 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-12 15:23:40,085 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-12 15:23:40,086 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:40,086 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-12 15:23:40,087 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase5.apache.org,36017,1689175380873; numProcessing=1 2023-07-12 15:23:40,087 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-12 15:23:40,087 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=19, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,36017,1689175380873, splitWal=true, meta=false 2023-07-12 15:23:40,087 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase5.apache.org,35567,1689175380829; numProcessing=2 2023-07-12 15:23:40,087 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-12 15:23:40,087 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,35567,1689175380829, splitWal=true, meta=true 2023-07-12 15:23:40,088 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,35567,1689175380829, splitWal=true, meta=true, isMeta: true 2023-07-12 15:23:40,092 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-12 15:23:40,092 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-12 15:23:40,093 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,35567,1689175380829-splitting 2023-07-12 15:23:40,094 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,35567,1689175380829-splitting dir is empty, no logs to split. 2023-07-12 15:23:40,094 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase5.apache.org,35567,1689175380829 WAL count=0, meta=true 2023-07-12 15:23:40,095 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175420092,5,FailOnTimeoutGroup] 2023-07-12 15:23:40,096 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,35567,1689175380829-splitting dir is empty, no logs to split. 2023-07-12 15:23:40,096 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase5.apache.org,35567,1689175380829 WAL count=0, meta=true 2023-07-12 15:23:40,096 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase5.apache.org,35567,1689175380829 WAL splitting is done? wals=0, meta=true 2023-07-12 15:23:40,096 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175420095,5,FailOnTimeoutGroup] 2023-07-12 15:23:40,096 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:40,097 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-12 15:23:40,097 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:40,097 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:40,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-12 15:23:40,097 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689175420097, completionTime=-1 2023-07-12 15:23:40,097 WARN [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-12 15:23:40,097 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-12 15:23:40,098 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-12 15:23:40,099 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-12 15:23:40,105 INFO [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(951): ClusterId : 098cb589-347a-4654-bf7c-f2a1c06d1dde 2023-07-12 15:23:40,105 DEBUG [RS:0;jenkins-hbase5:43385] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 15:23:40,107 DEBUG [RS:0;jenkins-hbase5:43385] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 15:23:40,107 DEBUG [RS:0;jenkins-hbase5:43385] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 15:23:40,109 DEBUG [RS:0;jenkins-hbase5:43385] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 15:23:40,112 DEBUG [RS:0;jenkins-hbase5:43385] zookeeper.ReadOnlyZKClient(139): Connect 0x2fe9940b to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:40,116 DEBUG [RS:0;jenkins-hbase5:43385] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@212e6f86, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:40,116 DEBUG [RS:0;jenkins-hbase5:43385] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2511ff7b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase5.apache.org/172.31.10.131:0 2023-07-12 15:23:40,124 DEBUG [RS:0;jenkins-hbase5:43385] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase5:43385 2023-07-12 15:23:40,124 INFO [RS:0;jenkins-hbase5:43385] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 15:23:40,124 INFO [RS:0;jenkins-hbase5:43385] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 15:23:40,124 DEBUG [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 15:23:40,125 INFO [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase5.apache.org,33093,1689175419847 with isa=jenkins-hbase5.apache.org/172.31.10.131:43385, startcode=1689175419880 2023-07-12 15:23:40,125 DEBUG [RS:0;jenkins-hbase5:43385] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 15:23:40,127 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:49735, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 15:23:40,128 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33093] master.ServerManager(394): Registering regionserver=jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:40,129 DEBUG [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4 2023-07-12 15:23:40,129 DEBUG [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:41947 2023-07-12 15:23:40,129 DEBUG [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 15:23:40,130 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-12 15:23:40,131 DEBUG [RS:0;jenkins-hbase5:43385] zookeeper.ZKUtil(162): regionserver:43385-0x101c5bd87ea002b, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:40,132 WARN [RS:0;jenkins-hbase5:43385] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:23:40,132 INFO [RS:0;jenkins-hbase5:43385] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:40,132 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase5.apache.org,43385,1689175419880] 2023-07-12 15:23:40,132 DEBUG [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(1948): logDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:40,147 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-12 15:23:40,149 INFO [RS:0;jenkins-hbase5:43385] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-12 15:23:40,150 DEBUG [RS:0;jenkins-hbase5:43385] zookeeper.ZKUtil(162): regionserver:43385-0x101c5bd87ea002b, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:40,151 DEBUG [RS:0;jenkins-hbase5:43385] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-12 15:23:40,151 INFO [RS:0;jenkins-hbase5:43385] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 15:23:40,152 INFO [RS:0;jenkins-hbase5:43385] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 15:23:40,153 INFO [RS:0;jenkins-hbase5:43385] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 15:23:40,153 INFO [RS:0;jenkins-hbase5:43385] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:40,155 INFO [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 15:23:40,157 INFO [RS:0;jenkins-hbase5:43385] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:40,157 DEBUG [RS:0;jenkins-hbase5:43385] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:40,157 DEBUG [RS:0;jenkins-hbase5:43385] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:40,157 DEBUG [RS:0;jenkins-hbase5:43385] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:40,157 DEBUG [RS:0;jenkins-hbase5:43385] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:40,157 DEBUG [RS:0;jenkins-hbase5:43385] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:40,157 DEBUG [RS:0;jenkins-hbase5:43385] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase5:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:23:40,157 DEBUG [RS:0;jenkins-hbase5:43385] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:40,157 DEBUG [RS:0;jenkins-hbase5:43385] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:40,157 DEBUG [RS:0;jenkins-hbase5:43385] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:40,157 DEBUG [RS:0;jenkins-hbase5:43385] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:40,159 INFO [RS:0;jenkins-hbase5:43385] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:40,159 INFO [RS:0;jenkins-hbase5:43385] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:40,159 INFO [RS:0;jenkins-hbase5:43385] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:40,171 INFO [RS:0;jenkins-hbase5:43385] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 15:23:40,171 INFO [RS:0;jenkins-hbase5:43385] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,43385,1689175419880-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:40,181 INFO [RS:0;jenkins-hbase5:43385] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-12 15:23:40,182 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [1689175399498.syncuptool.replication.org,1234,1] other RSs: [jenkins-hbase5.apache.org,43385,1689175419880] 2023-07-12 15:23:40,193 INFO [RS:0;jenkins-hbase5:43385] regionserver.Replication(203): jenkins-hbase5.apache.org,43385,1689175419880 started 2023-07-12 15:23:40,193 INFO [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(1637): Serving as jenkins-hbase5.apache.org,43385,1689175419880, RpcServer on jenkins-hbase5.apache.org/172.31.10.131:43385, sessionid=0x101c5bd87ea002b 2023-07-12 15:23:40,193 DEBUG [RS:0;jenkins-hbase5:43385] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 15:23:40,193 DEBUG [RS:0;jenkins-hbase5:43385] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:40,193 DEBUG [RS:0;jenkins-hbase5:43385] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,43385,1689175419880' 2023-07-12 15:23:40,193 DEBUG [RS:0;jenkins-hbase5:43385] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-12 15:23:40,193 DEBUG [RS:0;jenkins-hbase5:43385] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-12 15:23:40,194 DEBUG [RS:0;jenkins-hbase5:43385] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 15:23:40,194 DEBUG [RS:0;jenkins-hbase5:43385] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 15:23:40,194 DEBUG [RS:0;jenkins-hbase5:43385] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:40,194 DEBUG [RS:0;jenkins-hbase5:43385] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,43385,1689175419880' 2023-07-12 15:23:40,194 DEBUG [RS:0;jenkins-hbase5:43385] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-12 15:23:40,194 DEBUG [RS:0;jenkins-hbase5:43385] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-12 15:23:40,194 DEBUG [RS:0;jenkins-hbase5:43385] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 15:23:40,194 INFO [RS:0;jenkins-hbase5:43385] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 15:23:40,195 INFO [RS:0;jenkins-hbase5:43385] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 15:23:40,203 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x74ab4ca3 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:40,209 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@71bd3ffe, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:40,209 INFO [RS:0;jenkins-hbase5:43385.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:40,211 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:40,214 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101c5bd87ea002f connected 2023-07-12 15:23:40,214 INFO [RS:0;jenkins-hbase5:43385.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=098cb589-347a-4654-bf7c-f2a1c06d1dde to cluster=5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:23:40,250 DEBUG [jenkins-hbase5:33093] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-12 15:23:40,251 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase5.apache.org,43385,1689175419880, state=OPENING 2023-07-12 15:23:40,252 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-12 15:23:40,252 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase5.apache.org,43385,1689175419880}] 2023-07-12 15:23:40,252 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-12 15:23:40,296 INFO [RS:0;jenkins-hbase5:43385] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C43385%2C1689175419880, suffix=, logDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,43385,1689175419880, archiveDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs, maxLogs=10 2023-07-12 15:23:40,313 DEBUG [RS-EventLoopGroup-15-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38179,DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770,DISK] 2023-07-12 15:23:40,313 DEBUG [RS-EventLoopGroup-15-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34197,DS-999b06e9-279a-47df-944d-eabe177df1c6,DISK] 2023-07-12 15:23:40,321 DEBUG [RS:0;jenkins-hbase5:43385] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase5.apache.org%2C43385%2C1689175419880 for peer 1 2023-07-12 15:23:40,321 INFO [RS:0;jenkins-hbase5:43385] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,43385,1689175419880/jenkins-hbase5.apache.org%2C43385%2C1689175419880.1689175420297 2023-07-12 15:23:40,321 DEBUG [RS:0;jenkins-hbase5:43385] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase5.apache.org%2C43385%2C1689175419880 2023-07-12 15:23:40,321 INFO [RS:0;jenkins-hbase5:43385] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-12 15:23:40,322 DEBUG [RS:0;jenkins-hbase5:43385] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38179,DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770,DISK], DatanodeInfoWithStorage[127.0.0.1:34197,DS-999b06e9-279a-47df-944d-eabe177df1c6,DISK]] 2023-07-12 15:23:40,322 INFO [RS:0;jenkins-hbase5:43385.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase5.apache.org%2C43385%2C1689175419880 2023-07-12 15:23:40,327 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:40,408 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:40,409 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:23:40,410 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:36592, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 15:23:40,414 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-12 15:23:40,414 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:40,415 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C43385%2C1689175419880.meta, suffix=.meta, logDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,43385,1689175419880, archiveDir=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs, maxLogs=10 2023-07-12 15:23:40,429 DEBUG [RS-EventLoopGroup-15-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34197,DS-999b06e9-279a-47df-944d-eabe177df1c6,DISK] 2023-07-12 15:23:40,429 DEBUG [RS-EventLoopGroup-15-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38179,DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770,DISK] 2023-07-12 15:23:40,431 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,43385,1689175419880/jenkins-hbase5.apache.org%2C43385%2C1689175419880.meta.1689175420415.meta 2023-07-12 15:23:40,431 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38179,DS-2fecc2e4-7dca-4c8d-84ed-e7af75e14770,DISK], DatanodeInfoWithStorage[127.0.0.1:34197,DS-999b06e9-279a-47df-944d-eabe177df1c6,DISK]] 2023-07-12 15:23:40,432 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:40,432 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-12 15:23:40,432 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-12 15:23:40,432 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-12 15:23:40,432 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:40,432 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-12 15:23:40,432 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:40,432 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-12 15:23:40,432 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-12 15:23:40,434 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-12 15:23:40,434 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/info 2023-07-12 15:23:40,435 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/info 2023-07-12 15:23:40,435 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-12 15:23:40,441 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/info/08bc8a0183c64c54b0ce393f29c1440b 2023-07-12 15:23:40,441 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:40,441 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-12 15:23:40,442 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:23:40,443 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:23:40,443 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-12 15:23:40,449 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/rep_barrier/3699d38d4fe144b4b4117dfd4f0b5c51 2023-07-12 15:23:40,449 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:40,450 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-12 15:23:40,451 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/table 2023-07-12 15:23:40,451 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/table 2023-07-12 15:23:40,451 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-12 15:23:40,458 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/table/452d4cba21c1453fa96f77780c0f84eb 2023-07-12 15:23:40,458 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:40,459 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740 2023-07-12 15:23:40,460 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740 2023-07-12 15:23:40,463 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-12 15:23:40,465 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-12 15:23:40,466 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62356042, jitterRate=-0.0708225667476654}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-12 15:23:40,466 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-12 15:23:40,467 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=21, masterSystemTime=1689175420408 2023-07-12 15:23:40,475 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-12 15:23:40,476 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-12 15:23:40,476 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase5.apache.org,43385,1689175419880, state=OPEN 2023-07-12 15:23:40,479 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-12 15:23:40,479 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-12 15:23:40,481 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-07-12 15:23:40,481 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase5.apache.org,43385,1689175419880 in 227 msec 2023-07-12 15:23:40,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=18 2023-07-12 15:23:40,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 384 msec 2023-07-12 15:23:40,537 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:40,642 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:40,747 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:40,851 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:40,956 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:41,061 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:41,166 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:41,272 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:41,376 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:41,481 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:41,586 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:41,649 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1552ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-07-12 15:23:41,690 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:41,795 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:41,899 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:42,004 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:42,108 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:42,213 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:42,317 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:42,422 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:42,526 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:42,631 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:42,735 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:42,840 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:42,944 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:43,049 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:43,152 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3055ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-12 15:23:43,155 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:43,259 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:43,366 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:43,472 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:43,474 INFO [jenkins-hbase5:40223Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:23:43,474 INFO [jenkins-hbase5:40223Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-12 15:23:43,577 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:43,681 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:43,786 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:43,890 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:43,995 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:44,099 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:44,203 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:44,308 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:44,413 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:44,517 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:44,604 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-12 15:23:44,604 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:23:44,609 DEBUG [hconnection-0x5e6742e1-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:44,610 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:36596, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:44,615 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=2f5120c49287770b2e8c9c01ffb12e34, regionState=OPEN, lastHost=jenkins-hbase5.apache.org,35567,1689175380829, regionLocation=jenkins-hbase5.apache.org,35567,1689175380829, openSeqNum=2 2023-07-12 15:23:44,616 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=e2a1d8488c64dcf2f22986998cf96932, regionState=OPEN, lastHost=jenkins-hbase5.apache.org,36017,1689175380873, regionLocation=jenkins-hbase5.apache.org,36017,1689175380873, openSeqNum=2 2023-07-12 15:23:44,616 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=2c7136e581575f5b2280ddc3c686997b, regionState=OPEN, lastHost=jenkins-hbase5.apache.org,36017,1689175380873, regionLocation=jenkins-hbase5.apache.org,36017,1689175380873, openSeqNum=2 2023-07-12 15:23:44,616 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-12 15:23:44,616 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689175484616 2023-07-12 15:23:44,616 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689175544616 2023-07-12 15:23:44,616 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 11 msec 2023-07-12 15:23:44,624 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:44,634 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase5.apache.org,35567,1689175380829 had 2 regions 2023-07-12 15:23:44,634 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,33093,1689175419847-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:44,634 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase5.apache.org,36017,1689175380873 had 2 regions 2023-07-12 15:23:44,634 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,33093,1689175419847-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:44,634 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,33093,1689175419847-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:44,634 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase5:33093, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:44,634 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:44,634 WARN [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. is NOT online; state={2f5120c49287770b2e8c9c01ffb12e34 state=OPEN, ts=1689175424615, server=jenkins-hbase5.apache.org,35567,1689175380829}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-12 15:23:44,635 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,35567,1689175380829, splitWal=true, meta=true, isMeta: false 2023-07-12 15:23:44,635 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,36017,1689175380873, splitWal=true, meta=false, isMeta: false 2023-07-12 15:23:44,638 WARN [master/jenkins-hbase5:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase5.apache.org,35567,1689175380829/hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34., unknown_server=jenkins-hbase5.apache.org,36017,1689175380873/t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932., unknown_server=jenkins-hbase5.apache.org,36017,1689175380873/t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:44,638 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873-splitting 2023-07-12 15:23:44,638 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,35567,1689175380829-splitting dir is empty, no logs to split. 2023-07-12 15:23:44,639 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase5.apache.org,35567,1689175380829 WAL count=0, meta=false 2023-07-12 15:23:44,639 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873-splitting dir is empty, no logs to split. 2023-07-12 15:23:44,640 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase5.apache.org,36017,1689175380873 WAL count=0, meta=false 2023-07-12 15:23:44,642 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,35567,1689175380829-splitting dir is empty, no logs to split. 2023-07-12 15:23:44,642 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase5.apache.org,35567,1689175380829 WAL count=0, meta=false 2023-07-12 15:23:44,642 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase5.apache.org,35567,1689175380829 WAL splitting is done? wals=0, meta=false 2023-07-12 15:23:44,642 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873-splitting dir is empty, no logs to split. 2023-07-12 15:23:44,642 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase5.apache.org,36017,1689175380873 WAL count=0, meta=false 2023-07-12 15:23:44,642 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase5.apache.org,36017,1689175380873 WAL splitting is done? wals=0, meta=false 2023-07-12 15:23:44,645 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase5.apache.org,36017,1689175380873 failed, ignore...File hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,36017,1689175380873-splitting does not exist. 2023-07-12 15:23:44,647 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=2f5120c49287770b2e8c9c01ffb12e34, ASSIGN}] 2023-07-12 15:23:44,648 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=2c7136e581575f5b2280ddc3c686997b, ASSIGN}, {pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=e2a1d8488c64dcf2f22986998cf96932, ASSIGN}] 2023-07-12 15:23:44,650 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=2f5120c49287770b2e8c9c01ffb12e34, ASSIGN 2023-07-12 15:23:44,651 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=2c7136e581575f5b2280ddc3c686997b, ASSIGN 2023-07-12 15:23:44,651 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=e2a1d8488c64dcf2f22986998cf96932, ASSIGN 2023-07-12 15:23:44,651 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=2f5120c49287770b2e8c9c01ffb12e34, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-12 15:23:44,652 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=2c7136e581575f5b2280ddc3c686997b, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-12 15:23:44,652 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=e2a1d8488c64dcf2f22986998cf96932, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-12 15:23:44,729 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:44,802 DEBUG [jenkins-hbase5:33093] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-12 15:23:44,805 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=2f5120c49287770b2e8c9c01ffb12e34, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:44,805 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=e2a1d8488c64dcf2f22986998cf96932, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:44,805 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=2c7136e581575f5b2280ddc3c686997b, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:44,805 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689175424805"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175424805"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175424805"}]},"ts":"1689175424805"} 2023-07-12 15:23:44,805 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175424805"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175424805"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175424805"}]},"ts":"1689175424805"} 2023-07-12 15:23:44,805 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175424805"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175424805"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175424805"}]},"ts":"1689175424805"} 2023-07-12 15:23:44,808 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure 2f5120c49287770b2e8c9c01ffb12e34, server=jenkins-hbase5.apache.org,43385,1689175419880}] 2023-07-12 15:23:44,808 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=24, state=RUNNABLE; OpenRegionProcedure e2a1d8488c64dcf2f22986998cf96932, server=jenkins-hbase5.apache.org,43385,1689175419880}] 2023-07-12 15:23:44,809 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=23, state=RUNNABLE; OpenRegionProcedure 2c7136e581575f5b2280ddc3c686997b, server=jenkins-hbase5.apache.org,43385,1689175419880}] 2023-07-12 15:23:44,833 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:44,938 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-12 15:23:44,963 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:44,964 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2f5120c49287770b2e8c9c01ffb12e34, NAME => 'hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:44,964 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:44,964 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:44,964 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:44,964 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:44,966 INFO [StoreOpener-2f5120c49287770b2e8c9c01ffb12e34-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:44,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:44,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2c7136e581575f5b2280ddc3c686997b, NAME => 't2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:44,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:44,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:44,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:44,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:44,967 DEBUG [StoreOpener-2f5120c49287770b2e8c9c01ffb12e34-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34/info 2023-07-12 15:23:44,967 DEBUG [StoreOpener-2f5120c49287770b2e8c9c01ffb12e34-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34/info 2023-07-12 15:23:44,967 INFO [StoreOpener-2f5120c49287770b2e8c9c01ffb12e34-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2f5120c49287770b2e8c9c01ffb12e34 columnFamilyName info 2023-07-12 15:23:44,968 INFO [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:44,969 DEBUG [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/cf1 2023-07-12 15:23:44,969 DEBUG [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/cf1 2023-07-12 15:23:44,969 INFO [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c7136e581575f5b2280ddc3c686997b columnFamilyName cf1 2023-07-12 15:23:44,975 DEBUG [StoreOpener-2f5120c49287770b2e8c9c01ffb12e34-1] regionserver.HStore(539): loaded hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34/info/5b1005ad38d547d98cb6dd0a418765c2 2023-07-12 15:23:44,975 INFO [StoreOpener-2f5120c49287770b2e8c9c01ffb12e34-1] regionserver.HStore(310): Store=2f5120c49287770b2e8c9c01ffb12e34/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:44,976 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 318232eb904e4b49acd994e031633b58 2023-07-12 15:23:44,976 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:44,976 DEBUG [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] regionserver.HStore(539): loaded hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/cf1/318232eb904e4b49acd994e031633b58 2023-07-12 15:23:44,977 INFO [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] regionserver.HStore(310): Store=2c7136e581575f5b2280ddc3c686997b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:44,977 INFO [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:44,978 DEBUG [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/norep 2023-07-12 15:23:44,978 DEBUG [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/norep 2023-07-12 15:23:44,978 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:44,978 INFO [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c7136e581575f5b2280ddc3c686997b columnFamilyName norep 2023-07-12 15:23:44,984 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:44,985 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 2f5120c49287770b2e8c9c01ffb12e34; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63096164, jitterRate=-0.05979388952255249}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:23:44,985 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 2f5120c49287770b2e8c9c01ffb12e34: 2023-07-12 15:23:44,986 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34., pid=25, masterSystemTime=1689175424960 2023-07-12 15:23:44,989 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6bcf7203f4f84b4a97b28d7d0ba402e4 2023-07-12 15:23:44,989 DEBUG [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] regionserver.HStore(539): loaded hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/norep/6bcf7203f4f84b4a97b28d7d0ba402e4 2023-07-12 15:23:44,989 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:44,989 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:44,989 INFO [StoreOpener-2c7136e581575f5b2280ddc3c686997b-1] regionserver.HStore(310): Store=2c7136e581575f5b2280ddc3c686997b/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:44,989 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=2f5120c49287770b2e8c9c01ffb12e34, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:44,990 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689175424989"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175424989"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175424989"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175424989"}]},"ts":"1689175424989"} 2023-07-12 15:23:44,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:44,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:44,993 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-07-12 15:23:44,993 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure 2f5120c49287770b2e8c9c01ffb12e34, server=jenkins-hbase5.apache.org,43385,1689175419880 in 183 msec 2023-07-12 15:23:44,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:23:44,994 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=18 2023-07-12 15:23:44,995 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=2f5120c49287770b2e8c9c01ffb12e34, ASSIGN in 346 msec 2023-07-12 15:23:44,995 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase5.apache.org,35567,1689175380829 after splitting done 2023-07-12 15:23:44,995 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase5.apache.org,35567,1689175380829 from processing; numProcessing=1 2023-07-12 15:23:44,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:44,996 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase5.apache.org,35567,1689175380829, splitWal=true, meta=true in 4.9520 sec 2023-07-12 15:23:44,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 2c7136e581575f5b2280ddc3c686997b; next sequenceid=210; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63303780, jitterRate=-0.056700170040130615}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:23:44,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 2c7136e581575f5b2280ddc3c686997b: 2023-07-12 15:23:44,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b., pid=27, masterSystemTime=1689175424960 2023-07-12 15:23:44,998 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=2c7136e581575f5b2280ddc3c686997b, regionState=OPEN, repBarrier=210, openSeqNum=210, regionLocation=jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:44,999 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175424998"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175424998"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175424998"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175424998"}]},"ts":"1689175424998"} 2023-07-12 15:23:44,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:44,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:44,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:44,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e2a1d8488c64dcf2f22986998cf96932, NAME => 't1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:45,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:45,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:45,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:45,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:45,001 INFO [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:45,002 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=23 2023-07-12 15:23:45,002 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=23, state=SUCCESS; OpenRegionProcedure 2c7136e581575f5b2280ddc3c686997b, server=jenkins-hbase5.apache.org,43385,1689175419880 in 191 msec 2023-07-12 15:23:45,002 DEBUG [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/cf1 2023-07-12 15:23:45,002 DEBUG [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/cf1 2023-07-12 15:23:45,003 INFO [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e2a1d8488c64dcf2f22986998cf96932 columnFamilyName cf1 2023-07-12 15:23:45,004 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=2c7136e581575f5b2280ddc3c686997b, ASSIGN in 354 msec 2023-07-12 15:23:45,010 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3749472cc82744ac9a6eaf3cb152ec28 2023-07-12 15:23:45,010 DEBUG [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] regionserver.HStore(539): loaded hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/cf1/3749472cc82744ac9a6eaf3cb152ec28 2023-07-12 15:23:45,010 INFO [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] regionserver.HStore(310): Store=e2a1d8488c64dcf2f22986998cf96932/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:45,010 INFO [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:45,011 DEBUG [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/norep 2023-07-12 15:23:45,011 DEBUG [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/norep 2023-07-12 15:23:45,011 INFO [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e2a1d8488c64dcf2f22986998cf96932 columnFamilyName norep 2023-07-12 15:23:45,018 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 161ea13628764952839a9efd2f670468 2023-07-12 15:23:45,018 DEBUG [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] regionserver.HStore(539): loaded hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/norep/161ea13628764952839a9efd2f670468 2023-07-12 15:23:45,018 INFO [StoreOpener-e2a1d8488c64dcf2f22986998cf96932-1] regionserver.HStore(310): Store=e2a1d8488c64dcf2f22986998cf96932/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:45,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:45,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:45,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:23:45,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:45,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened e2a1d8488c64dcf2f22986998cf96932; next sequenceid=110; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65887932, jitterRate=-0.018193304538726807}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:23:45,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for e2a1d8488c64dcf2f22986998cf96932: 2023-07-12 15:23:45,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932., pid=26, masterSystemTime=1689175424960 2023-07-12 15:23:45,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:45,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:45,027 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=e2a1d8488c64dcf2f22986998cf96932, regionState=OPEN, repBarrier=110, openSeqNum=110, regionLocation=jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:45,027 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175425027"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175425027"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175425027"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175425027"}]},"ts":"1689175425027"} 2023-07-12 15:23:45,030 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=24 2023-07-12 15:23:45,030 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=24, state=SUCCESS; OpenRegionProcedure e2a1d8488c64dcf2f22986998cf96932, server=jenkins-hbase5.apache.org,43385,1689175419880 in 220 msec 2023-07-12 15:23:45,032 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=19 2023-07-12 15:23:45,032 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase5.apache.org,36017,1689175380873 after splitting done 2023-07-12 15:23:45,032 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=e2a1d8488c64dcf2f22986998cf96932, ASSIGN in 382 msec 2023-07-12 15:23:45,032 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase5.apache.org,36017,1689175380873 from processing; numProcessing=0 2023-07-12 15:23:45,033 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; ServerCrashProcedure jenkins-hbase5.apache.org,36017,1689175380873, splitWal=true, meta=false in 4.9860 sec 2023-07-12 15:23:45,043 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-12 15:23:45,043 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-12 15:23:45,043 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-12 15:23:45,043 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,43385,1689175419880/jenkins-hbase5.apache.org%2C43385%2C1689175419880.1689175420297, lastWalPosition=1162, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-12 15:23:45,148 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-12 15:23:45,193 INFO [jenkins-hbase5:43385Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase5.apache.org%2C43385%2C1689175419880]: currently replicating from: hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,43385,1689175419880/jenkins-hbase5.apache.org%2C43385%2C1689175419880.1689175420297 at position: 1162 2023-07-12 15:23:45,202 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-12 15:23:45,256 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-12 15:23:45,361 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-12 15:23:45,465 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-12 15:23:45,570 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-12 15:23:45,635 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on existing znode=/1/namespace 2023-07-12 15:23:45,647 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-12 15:23:45,650 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-12 15:23:45,650 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.738sec 2023-07-12 15:23:45,650 INFO [master/jenkins-hbase5:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-12 15:23:45,650 INFO [master/jenkins-hbase5:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-12 15:23:45,650 INFO [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-12 15:23:45,650 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,33093,1689175419847-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-12 15:23:45,651 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,33093,1689175419847-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-12 15:23:45,653 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-12 15:23:45,674 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-12 15:23:45,710 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(139): Connect 0x3519f27f to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:45,714 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@41605d54, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:45,716 DEBUG [hconnection-0x4352b8f2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:45,717 INFO [RS-EventLoopGroup-15-1] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:36606, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:45,723 INFO [Listener at localhost/42735] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-12 15:23:45,723 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3519f27f to 127.0.0.1:64130 2023-07-12 15:23:45,724 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:45,725 INFO [Listener at localhost/42735] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase5.apache.org:33093 after: jenkins-hbase5.apache.org:33093 2023-07-12 15:23:45,726 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(139): Connect 0x0c3c78e3 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:45,734 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5affc4be, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:45,734 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1905cfa5 to 127.0.0.1:64130 2023-07-12 15:23:45,735 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:45,735 DEBUG [Listener at localhost/42735] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-12 15:23:45,735 DEBUG [Listener at localhost/42735] util.JVMClusterUtil(257): Found active master hash=1285807998, stopped=false 2023-07-12 15:23:45,735 INFO [Listener at localhost/42735] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase5.apache.org,39489,1689175392691 2023-07-12 15:23:45,736 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:40223-0x101c5bd87ea001d, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-12 15:23:45,736 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-12 15:23:45,736 INFO [Listener at localhost/42735] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:23:45,737 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:45,737 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x606ee338 to 127.0.0.1:64130 2023-07-12 15:23:45,738 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:45,738 INFO [Listener at localhost/42735] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase5.apache.org,40223,1689175392718' ***** 2023-07-12 15:23:45,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:45,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40223-0x101c5bd87ea001d, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:45,738 INFO [Listener at localhost/42735] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 15:23:45,739 INFO [RS:0;jenkins-hbase5:40223] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 15:23:45,739 INFO [RS:0;jenkins-hbase5:40223] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 15:23:45,739 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 15:23:45,739 INFO [RS:0;jenkins-hbase5:40223] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 15:23:45,740 INFO [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(3305): Received CLOSE for 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:45,740 INFO [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(3305): Received CLOSE for 19659d0923548261983bde115dd510a9 2023-07-12 15:23:45,740 INFO [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(3305): Received CLOSE for e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:45,740 INFO [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(1144): stopping server jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:45,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 144fbe8ee3e556340eac4ab1f11a0604, disabling compactions & flushes 2023-07-12 15:23:45,741 DEBUG [RS:0;jenkins-hbase5:40223] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x65d0dc9a to 127.0.0.1:64130 2023-07-12 15:23:45,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:45,741 DEBUG [RS:0;jenkins-hbase5:40223] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:45,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:45,741 INFO [RS:0;jenkins-hbase5:40223] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 15:23:45,741 INFO [RS:0;jenkins-hbase5:40223] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 15:23:45,741 INFO [RS:0;jenkins-hbase5:40223] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 15:23:45,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. after waiting 0 ms 2023-07-12 15:23:45,741 INFO [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-12 15:23:45,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:45,741 INFO [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-12 15:23:45,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing 144fbe8ee3e556340eac4ab1f11a0604 2/2 column families, dataSize=2.72 KB heapSize=10.66 KB 2023-07-12 15:23:45,741 DEBUG [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(1478): Online Regions={144fbe8ee3e556340eac4ab1f11a0604=t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604., 19659d0923548261983bde115dd510a9=t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9., 1588230740=hbase:meta,,1.1588230740, e7e73f29c14aa26ac2fc8049609eeb2d=hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.} 2023-07-12 15:23:45,742 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-12 15:23:45,742 DEBUG [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(1504): Waiting on 144fbe8ee3e556340eac4ab1f11a0604, 1588230740, 19659d0923548261983bde115dd510a9, e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:45,742 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-12 15:23:45,742 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-12 15:23:45,742 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-12 15:23:45,742 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-12 15:23:45,742 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-12 15:23:45,763 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/.tmp/cf1/4998f64c27364ed09f8f0dc657ca76fe 2023-07-12 15:23:45,763 INFO [regionserver/jenkins-hbase5:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:45,769 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/.tmp/info/2ba3e980bcde4b55b955f1761e409ce7 2023-07-12 15:23:45,772 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4998f64c27364ed09f8f0dc657ca76fe 2023-07-12 15:23:45,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/.tmp/cf1/4998f64c27364ed09f8f0dc657ca76fe as hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1/4998f64c27364ed09f8f0dc657ca76fe 2023-07-12 15:23:45,775 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/.tmp/info/2ba3e980bcde4b55b955f1761e409ce7 as hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info/2ba3e980bcde4b55b955f1761e409ce7 2023-07-12 15:23:45,780 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-12 15:23:45,782 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4998f64c27364ed09f8f0dc657ca76fe 2023-07-12 15:23:45,782 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1/4998f64c27364ed09f8f0dc657ca76fe, entries=100, sequenceid=14, filesize=7.9 K 2023-07-12 15:23:45,783 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2790, heapSize ~10.39 KB/10640, currentSize=0 B/0 for 144fbe8ee3e556340eac4ab1f11a0604 in 41ms, sequenceid=14, compaction requested=false 2023-07-12 15:23:45,784 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info/2ba3e980bcde4b55b955f1761e409ce7, entries=24, sequenceid=31, filesize=7.4 K 2023-07-12 15:23:45,785 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 43ms, sequenceid=31, compaction requested=false 2023-07-12 15:23:45,805 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=10 2023-07-12 15:23:45,807 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:45,807 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 144fbe8ee3e556340eac4ab1f11a0604: 2023-07-12 15:23:45,807 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:45,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 19659d0923548261983bde115dd510a9, disabling compactions & flushes 2023-07-12 15:23:45,808 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:45,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:45,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. after waiting 0 ms 2023-07-12 15:23:45,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:45,808 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing 19659d0923548261983bde115dd510a9 2/2 column families, dataSize=1.36 KB heapSize=5.58 KB 2023-07-12 15:23:45,809 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-12 15:23:45,810 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-12 15:23:45,811 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-12 15:23:45,811 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-12 15:23:45,811 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-12 15:23:45,822 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.36 KB at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/.tmp/cf1/d5c8c7910355405c8d6fb1542e12af8f 2023-07-12 15:23:45,828 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5c8c7910355405c8d6fb1542e12af8f 2023-07-12 15:23:45,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/.tmp/cf1/d5c8c7910355405c8d6fb1542e12af8f as hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1/d5c8c7910355405c8d6fb1542e12af8f 2023-07-12 15:23:45,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5c8c7910355405c8d6fb1542e12af8f 2023-07-12 15:23:45,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1/d5c8c7910355405c8d6fb1542e12af8f, entries=50, sequenceid=14, filesize=6.4 K 2023-07-12 15:23:45,835 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.36 KB/1390, heapSize ~5.31 KB/5440, currentSize=0 B/0 for 19659d0923548261983bde115dd510a9 in 27ms, sequenceid=14, compaction requested=false 2023-07-12 15:23:45,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=10 2023-07-12 15:23:45,845 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:45,845 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 19659d0923548261983bde115dd510a9: 2023-07-12 15:23:45,845 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:45,846 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing e7e73f29c14aa26ac2fc8049609eeb2d, disabling compactions & flushes 2023-07-12 15:23:45,846 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:45,846 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:45,846 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. after waiting 0 ms 2023-07-12 15:23:45,846 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:45,850 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-12 15:23:45,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:45,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for e7e73f29c14aa26ac2fc8049609eeb2d: 2023-07-12 15:23:45,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:45,885 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-12 15:23:45,942 INFO [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(1170): stopping server jenkins-hbase5.apache.org,40223,1689175392718; all regions closed. 2023-07-12 15:23:45,949 DEBUG [RS:0;jenkins-hbase5:40223] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs 2023-07-12 15:23:45,949 INFO [RS:0;jenkins-hbase5:40223] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase5.apache.org%2C40223%2C1689175392718.meta:.meta(num 1689175393664) 2023-07-12 15:23:45,954 DEBUG [RS:0;jenkins-hbase5:40223] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs 2023-07-12 15:23:45,954 INFO [RS:0;jenkins-hbase5:40223] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase5.apache.org%2C40223%2C1689175392718:(num 1689175393581) 2023-07-12 15:23:45,954 DEBUG [RS:0;jenkins-hbase5:40223] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:45,954 INFO [RS:0;jenkins-hbase5:40223] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:45,954 INFO [RS:0;jenkins-hbase5:40223] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase5:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-12 15:23:45,955 INFO [regionserver/jenkins-hbase5:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:23:45,955 DEBUG [RS:0;jenkins-hbase5:40223] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x05555d87 to 127.0.0.1:64130 2023-07-12 15:23:45,955 DEBUG [RS:0;jenkins-hbase5:40223] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:45,956 INFO [RS:0;jenkins-hbase5:40223] ipc.NettyRpcServer(158): Stopping server on /172.31.10.131:40223 2023-07-12 15:23:45,958 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:40223-0x101c5bd87ea001d, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase5.apache.org,40223,1689175392718 2023-07-12 15:23:45,958 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101c5bd87ea0026, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:45,958 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:40223-0x101c5bd87ea001d, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:45,958 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:45,958 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-12 15:23:45,959 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase5.apache.org,40223,1689175392718] 2023-07-12 15:23:45,959 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase5.apache.org,40223,1689175392718; numProcessing=1 2023-07-12 15:23:45,961 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase5.apache.org,40223,1689175392718 already deleted, retry=false 2023-07-12 15:23:45,961 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase5.apache.org,40223,1689175392718 expired; onlineServers=0 2023-07-12 15:23:45,961 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase5.apache.org,39489,1689175392691' ***** 2023-07-12 15:23:45,961 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-12 15:23:45,962 DEBUG [M:0;jenkins-hbase5:39489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@489c4232, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase5.apache.org/172.31.10.131:0 2023-07-12 15:23:45,962 INFO [M:0;jenkins-hbase5:39489] regionserver.HRegionServer(1144): stopping server jenkins-hbase5.apache.org,39489,1689175392691 2023-07-12 15:23:45,962 INFO [M:0;jenkins-hbase5:39489] regionserver.HRegionServer(1170): stopping server jenkins-hbase5.apache.org,39489,1689175392691; all regions closed. 2023-07-12 15:23:45,962 DEBUG [M:0;jenkins-hbase5:39489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:45,962 DEBUG [M:0;jenkins-hbase5:39489] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-12 15:23:45,962 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-12 15:23:45,962 DEBUG [M:0;jenkins-hbase5:39489] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-12 15:23:45,962 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175393324] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175393324,5,FailOnTimeoutGroup] 2023-07-12 15:23:45,963 INFO [M:0;jenkins-hbase5:39489] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-12 15:23:45,962 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175393324] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175393324,5,FailOnTimeoutGroup] 2023-07-12 15:23:45,963 INFO [M:0;jenkins-hbase5:39489] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-12 15:23:45,964 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-12 15:23:45,964 INFO [M:0;jenkins-hbase5:39489] hbase.ChoreService(369): Chore service for: master/jenkins-hbase5:0 had [] on shutdown 2023-07-12 15:23:45,964 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:45,964 DEBUG [M:0;jenkins-hbase5:39489] master.HMaster(1512): Stopping service threads 2023-07-12 15:23:45,964 INFO [M:0;jenkins-hbase5:39489] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-12 15:23:45,965 ERROR [M:0;jenkins-hbase5:39489] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:23:45,965 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-12 15:23:45,965 INFO [M:0;jenkins-hbase5:39489] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:23:45,965 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-12 15:23:45,965 DEBUG [M:0;jenkins-hbase5:39489] zookeeper.ZKUtil(398): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-12 15:23:45,965 WARN [M:0;jenkins-hbase5:39489] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-12 15:23:45,965 INFO [M:0;jenkins-hbase5:39489] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:23:45,966 INFO [M:0;jenkins-hbase5:39489] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-12 15:23:45,966 DEBUG [M:0;jenkins-hbase5:39489] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-12 15:23:45,966 INFO [M:0;jenkins-hbase5:39489] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:45,966 DEBUG [M:0;jenkins-hbase5:39489] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:45,967 DEBUG [M:0;jenkins-hbase5:39489] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-12 15:23:45,967 DEBUG [M:0;jenkins-hbase5:39489] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:45,967 INFO [M:0;jenkins-hbase5:39489] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=36.38 KB heapSize=46.77 KB 2023-07-12 15:23:45,979 INFO [M:0;jenkins-hbase5:39489] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=36.38 KB at sequenceid=238 (bloomFilter=true), to=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/fb4131ab8c434af580fd20da72e7e92a 2023-07-12 15:23:45,983 INFO [M:0;jenkins-hbase5:39489] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb4131ab8c434af580fd20da72e7e92a 2023-07-12 15:23:45,984 DEBUG [M:0;jenkins-hbase5:39489] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/fb4131ab8c434af580fd20da72e7e92a as hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/fb4131ab8c434af580fd20da72e7e92a 2023-07-12 15:23:45,989 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-12 15:23:45,990 INFO [M:0;jenkins-hbase5:39489] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb4131ab8c434af580fd20da72e7e92a 2023-07-12 15:23:45,990 INFO [M:0;jenkins-hbase5:39489] regionserver.HStore(1080): Added hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/fb4131ab8c434af580fd20da72e7e92a, entries=20, sequenceid=238, filesize=5.6 K 2023-07-12 15:23:45,990 INFO [M:0;jenkins-hbase5:39489] regionserver.HRegion(2948): Finished flush of dataSize ~36.38 KB/37256, heapSize ~46.75 KB/47872, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 23ms, sequenceid=238, compaction requested=false 2023-07-12 15:23:45,992 INFO [M:0;jenkins-hbase5:39489] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:45,992 DEBUG [M:0;jenkins-hbase5:39489] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:23:45,996 INFO [M:0;jenkins-hbase5:39489] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-12 15:23:45,996 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:23:45,997 INFO [M:0;jenkins-hbase5:39489] ipc.NettyRpcServer(158): Stopping server on /172.31.10.131:39489 2023-07-12 15:23:46,000 DEBUG [M:0;jenkins-hbase5:39489] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase5.apache.org,39489,1689175392691 already deleted, retry=false 2023-07-12 15:23:46,093 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-12 15:23:46,197 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-12 15:23:46,228 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:46,228 INFO [M:0;jenkins-hbase5:39489] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase5.apache.org,39489,1689175392691; zookeeper connection closed. 2023-07-12 15:23:46,228 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:39489-0x101c5bd87ea001c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:46,302 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 1162 2023-07-12 15:23:46,328 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:40223-0x101c5bd87ea001d, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:46,328 INFO [RS:0;jenkins-hbase5:40223] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase5.apache.org,40223,1689175392718; zookeeper connection closed. 2023-07-12 15:23:46,328 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:40223-0x101c5bd87ea001d, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:46,330 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1ecff213] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1ecff213 2023-07-12 15:23:46,330 INFO [Listener at localhost/42735] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-12 15:23:46,332 DEBUG [hconnection-0x43e04108-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:46,333 INFO [RS-EventLoopGroup-15-3] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:44808, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:46,413 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 8334 2023-07-12 15:23:46,413 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] regionserver.ReplicationSourceWALReader(162): Read 68 WAL entries eligible for replication 2023-07-12 15:23:46,413 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 8334 2023-07-12 15:23:46,413 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/e2a1d8488c64dcf2f22986998cf96932/112=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/113=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/114=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/115=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/116=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/117=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/118=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/119=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/120=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/121=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/122=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/123=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/124=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/125=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/126=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/127=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/128=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/129=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/130=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/131=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/132=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/133=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/134=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/135=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/136=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/137=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/138=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/139=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/140=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/141=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/142=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/143=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/144=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/145=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/146=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/147=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/148=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/149=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/150=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/151=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/152=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/153=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/154=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/155=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/156=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/157=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/158=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/159=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/160=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/161=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/162=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/163=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/164=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/165=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/166=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/167=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/168=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/169=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/170=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/171=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/172=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/173=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/174=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/175=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/176=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/177=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/178=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/179=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,43385,1689175419880/jenkins-hbase5.apache.org%2C43385%2C1689175419880.1689175420297, lastWalPosition=8334, nbRowKeys=68, nbHFiles=0, heapSize=14144, lastSeqIds={}, endOfFile=false,usedBufferSize=14144] 2023-07-12 15:23:46,417 WARN [RS:0;jenkins-hbase5:43385.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-12 15:23:46,517 WARN [RS:0;jenkins-hbase5:43385.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-12 15:23:46,524 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 21252 2023-07-12 15:23:46,525 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] regionserver.ReplicationSourceWALReader(162): Read 121 WAL entries eligible for replication 2023-07-12 15:23:46,525 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 21252 2023-07-12 15:23:46,622 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0c3c78e3 to 127.0.0.1:64130 2023-07-12 15:23:46,623 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:46,623 DEBUG [Listener at localhost/42735] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-12 15:23:46,623 DEBUG [Listener at localhost/42735] util.JVMClusterUtil(257): Found active master hash=439655823, stopped=false 2023-07-12 15:23:46,623 INFO [Listener at localhost/42735] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase5.apache.org,33093,1689175419847 2023-07-12 15:23:46,625 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-12 15:23:46,625 INFO [Listener at localhost/42735] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:23:46,626 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:46,625 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:43385-0x101c5bd87ea002b, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-12 15:23:46,626 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-12 15:23:46,627 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1a289340 to 127.0.0.1:64130 2023-07-12 15:23:46,628 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:46,629 INFO [Listener at localhost/42735] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase5.apache.org,43385,1689175419880' ***** 2023-07-12 15:23:46,629 INFO [Listener at localhost/42735] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 15:23:46,629 INFO [RS:0;jenkins-hbase5:43385] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 15:23:46,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43385-0x101c5bd87ea002b, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-12 15:23:46,629 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 15:23:46,629 INFO [RS:0;jenkins-hbase5:43385] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 15:23:46,630 INFO [RS:0;jenkins-hbase5:43385] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 15:23:46,630 INFO [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(3305): Received CLOSE for 2c7136e581575f5b2280ddc3c686997b 2023-07-12 15:23:46,630 INFO [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(3305): Received CLOSE for 2f5120c49287770b2e8c9c01ffb12e34 2023-07-12 15:23:46,630 INFO [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(3305): Received CLOSE for e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:46,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 2c7136e581575f5b2280ddc3c686997b, disabling compactions & flushes 2023-07-12 15:23:46,631 INFO [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(1144): stopping server jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:46,631 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:46,631 DEBUG [RS:0;jenkins-hbase5:43385] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2fe9940b to 127.0.0.1:64130 2023-07-12 15:23:46,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:46,631 DEBUG [RS:0;jenkins-hbase5:43385] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:46,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. after waiting 0 ms 2023-07-12 15:23:46,631 INFO [RS:0;jenkins-hbase5:43385] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 15:23:46,631 INFO [RS:0;jenkins-hbase5:43385] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 15:23:46,631 INFO [RS:0;jenkins-hbase5:43385] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 15:23:46,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:46,631 INFO [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-12 15:23:46,631 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing 2c7136e581575f5b2280ddc3c686997b 2/2 column families, dataSize=7.05 KB heapSize=22.49 KB 2023-07-12 15:23:46,631 INFO [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-12 15:23:46,632 DEBUG [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(1478): Online Regions={2c7136e581575f5b2280ddc3c686997b=t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b., 2f5120c49287770b2e8c9c01ffb12e34=hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34., 1588230740=hbase:meta,,1.1588230740, e2a1d8488c64dcf2f22986998cf96932=t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932.} 2023-07-12 15:23:46,632 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-12 15:23:46,632 DEBUG [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(1504): Waiting on 1588230740, 2c7136e581575f5b2280ddc3c686997b, 2f5120c49287770b2e8c9c01ffb12e34, e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:46,632 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-12 15:23:46,632 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-12 15:23:46,632 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-12 15:23:46,632 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-12 15:23:46,633 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.94 KB heapSize=5.62 KB 2023-07-12 15:23:46,638 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] wal.ProtobufLogReader(420): EOF at position 33330 2023-07-12 15:23:46,638 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1] regionserver.ReplicationSourceWALReader(162): Read 111 WAL entries eligible for replication 2023-07-12 15:23:46,655 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/.tmp/info/90dd15a9e26a42de8da9b6b36ee9550d 2023-07-12 15:23:46,664 INFO [regionserver/jenkins-hbase5:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:46,674 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/.tmp/rep_barrier/75a1a2ac4d3d4acb8c1a1e40cdd2e83e 2023-07-12 15:23:46,679 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/.tmp/info/90dd15a9e26a42de8da9b6b36ee9550d as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/info/90dd15a9e26a42de8da9b6b36ee9550d 2023-07-12 15:23:46,685 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/info/90dd15a9e26a42de8da9b6b36ee9550d, entries=24, sequenceid=31, filesize=7.4 K 2023-07-12 15:23:46,686 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/.tmp/rep_barrier/75a1a2ac4d3d4acb8c1a1e40cdd2e83e as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/rep_barrier/75a1a2ac4d3d4acb8c1a1e40cdd2e83e 2023-07-12 15:23:46,691 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/rep_barrier/75a1a2ac4d3d4acb8c1a1e40cdd2e83e, entries=2, sequenceid=31, filesize=4.9 K 2023-07-12 15:23:46,692 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.94 KB/3012, heapSize ~5.34 KB/5464, currentSize=0 B/0 for 1588230740 in 59ms, sequenceid=31, compaction requested=false 2023-07-12 15:23:46,700 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-12 15:23:46,700 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-12 15:23:46,701 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-12 15:23:46,701 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-12 15:23:46,701 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-12 15:23:46,718 WARN [RS:0;jenkins-hbase5:43385.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-12 15:23:46,832 DEBUG [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(1504): Waiting on 2c7136e581575f5b2280ddc3c686997b, 2f5120c49287770b2e8c9c01ffb12e34, e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:47,019 WARN [RS:0;jenkins-hbase5:43385.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-12 15:23:47,033 DEBUG [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(1504): Waiting on 2c7136e581575f5b2280ddc3c686997b, 2f5120c49287770b2e8c9c01ffb12e34, e2a1d8488c64dcf2f22986998cf96932 2023-07-12 15:23:47,055 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/.tmp/cf1/2de52c94e45744ed8266370647363b48 2023-07-12 15:23:47,070 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/.tmp/norep/248be710f91e44af94f5f728b9635b10 2023-07-12 15:23:47,075 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/.tmp/cf1/2de52c94e45744ed8266370647363b48 as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/cf1/2de52c94e45744ed8266370647363b48 2023-07-12 15:23:47,080 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/cf1/2de52c94e45744ed8266370647363b48, entries=200, sequenceid=413, filesize=12.5 K 2023-07-12 15:23:47,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/.tmp/norep/248be710f91e44af94f5f728b9635b10 as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/norep/248be710f91e44af94f5f728b9635b10 2023-07-12 15:23:47,085 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/norep/248be710f91e44af94f5f728b9635b10, entries=1, sequenceid=413, filesize=4.8 K 2023-07-12 15:23:47,086 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.05 KB/7221, heapSize ~22.46 KB/23000, currentSize=0 B/0 for 2c7136e581575f5b2280ddc3c686997b in 455ms, sequenceid=413, compaction requested=false 2023-07-12 15:23:47,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t2_syncup/2c7136e581575f5b2280ddc3c686997b/recovered.edits/416.seqid, newMaxSeqId=416, maxSeqId=209 2023-07-12 15:23:47,094 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:47,094 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 2c7136e581575f5b2280ddc3c686997b: 2023-07-12 15:23:47,094 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689175386902.2c7136e581575f5b2280ddc3c686997b. 2023-07-12 15:23:47,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 2f5120c49287770b2e8c9c01ffb12e34, disabling compactions & flushes 2023-07-12 15:23:47,095 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:47,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:47,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. after waiting 0 ms 2023-07-12 15:23:47,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:47,098 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/hbase/namespace/2f5120c49287770b2e8c9c01ffb12e34/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-12 15:23:47,099 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:47,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 2f5120c49287770b2e8c9c01ffb12e34: 2023-07-12 15:23:47,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689175382758.2f5120c49287770b2e8c9c01ffb12e34. 2023-07-12 15:23:47,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing e2a1d8488c64dcf2f22986998cf96932, disabling compactions & flushes 2023-07-12 15:23:47,099 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:47,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:47,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. after waiting 0 ms 2023-07-12 15:23:47,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:47,099 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing e2a1d8488c64dcf2f22986998cf96932 2/2 column families, dataSize=3.44 KB heapSize=11.55 KB 2023-07-12 15:23:47,108 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/.tmp/cf1/558b5936b78842f295fa7da99b61d1cb 2023-07-12 15:23:47,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/.tmp/norep/5b62df3d1592409992eb87696dd0885b 2023-07-12 15:23:47,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/.tmp/cf1/558b5936b78842f295fa7da99b61d1cb as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/cf1/558b5936b78842f295fa7da99b61d1cb 2023-07-12 15:23:47,131 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/cf1/558b5936b78842f295fa7da99b61d1cb, entries=100, sequenceid=213, filesize=8.4 K 2023-07-12 15:23:47,132 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/.tmp/norep/5b62df3d1592409992eb87696dd0885b as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/norep/5b62df3d1592409992eb87696dd0885b 2023-07-12 15:23:47,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/norep/5b62df3d1592409992eb87696dd0885b, entries=1, sequenceid=213, filesize=4.8 K 2023-07-12 15:23:47,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.44 KB/3521, heapSize ~11.52 KB/11800, currentSize=0 B/0 for e2a1d8488c64dcf2f22986998cf96932 in 38ms, sequenceid=213, compaction requested=false 2023-07-12 15:23:47,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/data/default/t1_syncup/e2a1d8488c64dcf2f22986998cf96932/recovered.edits/216.seqid, newMaxSeqId=216, maxSeqId=109 2023-07-12 15:23:47,145 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:47,145 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for e2a1d8488c64dcf2f22986998cf96932: 2023-07-12 15:23:47,145 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689175386263.e2a1d8488c64dcf2f22986998cf96932. 2023-07-12 15:23:47,159 INFO [regionserver/jenkins-hbase5:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-12 15:23:47,159 INFO [regionserver/jenkins-hbase5:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-12 15:23:47,233 INFO [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(1170): stopping server jenkins-hbase5.apache.org,43385,1689175419880; all regions closed. 2023-07-12 15:23:47,236 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/WALs/jenkins-hbase5.apache.org,43385,1689175419880/jenkins-hbase5.apache.org%2C43385%2C1689175419880.meta.1689175420415.meta not finished, retry = 0 2023-07-12 15:23:47,339 DEBUG [RS:0;jenkins-hbase5:43385] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs 2023-07-12 15:23:47,339 INFO [RS:0;jenkins-hbase5:43385] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase5.apache.org%2C43385%2C1689175419880.meta:.meta(num 1689175420415) 2023-07-12 15:23:47,345 DEBUG [RS:0;jenkins-hbase5:43385] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs 2023-07-12 15:23:47,345 INFO [RS:0;jenkins-hbase5:43385] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase5.apache.org%2C43385%2C1689175419880:(num 1689175420297) 2023-07-12 15:23:47,345 DEBUG [RS:0;jenkins-hbase5:43385] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:47,345 INFO [RS:0;jenkins-hbase5:43385] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:23:47,345 INFO [RS:0;jenkins-hbase5:43385] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase5:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-12 15:23:47,345 INFO [regionserver/jenkins-hbase5:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:23:47,345 INFO [RS:0;jenkins-hbase5:43385] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-12 15:23:47,419 WARN [RS:0;jenkins-hbase5:43385.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1] regionserver.HBaseInterClusterReplicationEndpoint(515): No replication sinks found, returning without replicating. The source should retry with the same set of edits. Not logging this again for the next 10 seconds. 2023-07-12 15:23:47,449 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101c5bd87ea002f, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:47,449 DEBUG [RS:0;jenkins-hbase5:43385] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x74ab4ca3 to 127.0.0.1:64130 2023-07-12 15:23:47,449 DEBUG [RS:0;jenkins-hbase5:43385.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101c5bd87ea002f, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:47,449 DEBUG [RS:0;jenkins-hbase5:43385] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:47,550 INFO [RS:0;jenkins-hbase5:43385] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS:0;jenkins-hbase5:43385.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1 terminated 2023-07-12 15:23:47,552 INFO [RS:0;jenkins-hbase5:43385] ipc.NettyRpcServer(158): Stopping server on /172.31.10.131:43385 2023-07-12 15:23:47,557 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-12 15:23:47,557 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:43385-0x101c5bd87ea002b, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:23:47,557 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:43385-0x101c5bd87ea002b, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-12 15:23:47,558 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase5.apache.org,43385,1689175419880] 2023-07-12 15:23:47,558 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase5.apache.org,43385,1689175419880; numProcessing=1 2023-07-12 15:23:47,560 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase5.apache.org,43385,1689175419880 already deleted, retry=false 2023-07-12 15:23:47,561 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase5.apache.org,43385,1689175419880 expired; onlineServers=0 2023-07-12 15:23:47,561 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase5.apache.org,33093,1689175419847' ***** 2023-07-12 15:23:47,561 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-12 15:23:47,561 DEBUG [M:0;jenkins-hbase5:33093] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1e0b0a11, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase5.apache.org/172.31.10.131:0 2023-07-12 15:23:47,561 INFO [M:0;jenkins-hbase5:33093] regionserver.HRegionServer(1144): stopping server jenkins-hbase5.apache.org,33093,1689175419847 2023-07-12 15:23:47,561 INFO [M:0;jenkins-hbase5:33093] regionserver.HRegionServer(1170): stopping server jenkins-hbase5.apache.org,33093,1689175419847; all regions closed. 2023-07-12 15:23:47,561 DEBUG [M:0;jenkins-hbase5:33093] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:47,561 DEBUG [M:0;jenkins-hbase5:33093] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-12 15:23:47,561 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-12 15:23:47,561 DEBUG [M:0;jenkins-hbase5:33093] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-12 15:23:47,562 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175420095] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175420095,5,FailOnTimeoutGroup] 2023-07-12 15:23:47,562 INFO [M:0;jenkins-hbase5:33093] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-12 15:23:47,562 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175420092] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175420092,5,FailOnTimeoutGroup] 2023-07-12 15:23:47,563 INFO [M:0;jenkins-hbase5:33093] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-12 15:23:47,563 INFO [M:0;jenkins-hbase5:33093] hbase.ChoreService(369): Chore service for: master/jenkins-hbase5:0 had [] on shutdown 2023-07-12 15:23:47,563 DEBUG [M:0;jenkins-hbase5:33093] master.HMaster(1512): Stopping service threads 2023-07-12 15:23:47,563 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-12 15:23:47,563 INFO [M:0;jenkins-hbase5:33093] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-12 15:23:47,563 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-12 15:23:47,564 ERROR [M:0;jenkins-hbase5:33093] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:23:47,564 INFO [M:0;jenkins-hbase5:33093] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:23:47,564 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-12 15:23:47,564 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-12 15:23:47,564 DEBUG [M:0;jenkins-hbase5:33093] zookeeper.ZKUtil(398): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-12 15:23:47,564 WARN [M:0;jenkins-hbase5:33093] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-12 15:23:47,564 INFO [M:0;jenkins-hbase5:33093] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:23:47,565 INFO [M:0;jenkins-hbase5:33093] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-12 15:23:47,567 DEBUG [M:0;jenkins-hbase5:33093] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-12 15:23:47,567 INFO [M:0;jenkins-hbase5:33093] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:47,567 DEBUG [M:0;jenkins-hbase5:33093] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:47,567 DEBUG [M:0;jenkins-hbase5:33093] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-12 15:23:47,567 DEBUG [M:0;jenkins-hbase5:33093] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:47,567 INFO [M:0;jenkins-hbase5:33093] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=29.71 KB heapSize=36.94 KB 2023-07-12 15:23:47,577 INFO [M:0;jenkins-hbase5:33093] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=29.71 KB at sequenceid=218 (bloomFilter=true), to=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/fed518f2b65746a1854e96e94e2126d5 2023-07-12 15:23:47,582 DEBUG [M:0;jenkins-hbase5:33093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/fed518f2b65746a1854e96e94e2126d5 as hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/fed518f2b65746a1854e96e94e2126d5 2023-07-12 15:23:47,586 INFO [M:0;jenkins-hbase5:33093] regionserver.HStore(1080): Added hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/fed518f2b65746a1854e96e94e2126d5, entries=10, sequenceid=218, filesize=5.8 K 2023-07-12 15:23:47,587 INFO [M:0;jenkins-hbase5:33093] regionserver.HRegion(2948): Finished flush of dataSize ~29.71 KB/30420, heapSize ~36.92 KB/37808, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=218, compaction requested=false 2023-07-12 15:23:47,589 INFO [M:0;jenkins-hbase5:33093] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:23:47,589 DEBUG [M:0;jenkins-hbase5:33093] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:23:47,593 INFO [M:0;jenkins-hbase5:33093] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-12 15:23:47,593 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:23:47,593 INFO [M:0;jenkins-hbase5:33093] ipc.NettyRpcServer(158): Stopping server on /172.31.10.131:33093 2023-07-12 15:23:47,595 DEBUG [M:0;jenkins-hbase5:33093] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase5.apache.org,33093,1689175419847 already deleted, retry=false 2023-07-12 15:23:47,659 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:43385-0x101c5bd87ea002b, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:47,659 INFO [RS:0;jenkins-hbase5:43385] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase5.apache.org,43385,1689175419880; zookeeper connection closed. 2023-07-12 15:23:47,659 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:43385-0x101c5bd87ea002b, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:47,660 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@47dee859] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@47dee859 2023-07-12 15:23:47,660 INFO [Listener at localhost/42735] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-12 15:23:47,759 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:47,759 INFO [M:0;jenkins-hbase5:33093] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase5.apache.org,33093,1689175419847; zookeeper connection closed. 2023-07-12 15:23:47,759 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:33093-0x101c5bd87ea002a, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:23:47,763 INFO [Listener at localhost/42735] client.ConnectionUtils(127): master/jenkins-hbase5:0 server-side Connection retries=18 2023-07-12 15:23:47,763 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:47,763 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:47,763 INFO [Listener at localhost/42735] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:23:47,763 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:47,763 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:23:47,763 INFO [Listener at localhost/42735] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:23:47,764 INFO [Listener at localhost/42735] ipc.NettyRpcServer(120): Bind to /172.31.10.131:43095 2023-07-12 15:23:47,764 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:47,765 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:47,766 INFO [Listener at localhost/42735] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43095 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:47,770 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:430950x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:47,771 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:43095-0x101c5bd87ea0032 connected 2023-07-12 15:23:47,773 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-12 15:23:47,773 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:47,774 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-12 15:23:47,774 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43095 2023-07-12 15:23:47,775 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43095 2023-07-12 15:23:47,779 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43095 2023-07-12 15:23:47,783 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43095 2023-07-12 15:23:47,783 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43095 2023-07-12 15:23:47,783 INFO [Listener at localhost/42735] master.HMaster(444): hbase.rootdir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143, hbase.cluster.distributed=false 2023-07-12 15:23:47,784 DEBUG [pool-638-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-12 15:23:47,796 INFO [Listener at localhost/42735] client.ConnectionUtils(127): regionserver/jenkins-hbase5:0 server-side Connection retries=18 2023-07-12 15:23:47,796 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:47,796 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:47,796 INFO [Listener at localhost/42735] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:23:47,796 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:23:47,796 INFO [Listener at localhost/42735] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:23:47,796 INFO [Listener at localhost/42735] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:23:47,797 INFO [Listener at localhost/42735] ipc.NettyRpcServer(120): Bind to /172.31.10.131:33709 2023-07-12 15:23:47,797 INFO [Listener at localhost/42735] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 15:23:47,798 DEBUG [Listener at localhost/42735] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 15:23:47,799 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:47,800 INFO [Listener at localhost/42735] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:47,801 INFO [Listener at localhost/42735] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33709 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:47,803 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:337090x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:47,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:33709-0x101c5bd87ea0033 connected 2023-07-12 15:23:47,805 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:33709-0x101c5bd87ea0033, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-12 15:23:47,805 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:33709-0x101c5bd87ea0033, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:23:47,806 DEBUG [Listener at localhost/42735] zookeeper.ZKUtil(164): regionserver:33709-0x101c5bd87ea0033, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-12 15:23:47,807 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33709 2023-07-12 15:23:47,807 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33709 2023-07-12 15:23:47,808 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33709 2023-07-12 15:23:47,808 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33709 2023-07-12 15:23:47,808 DEBUG [Listener at localhost/42735] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33709 2023-07-12 15:23:47,809 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase5.apache.org,43095,1689175427762 2023-07-12 15:23:47,811 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-12 15:23:47,811 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase5.apache.org,43095,1689175427762 2023-07-12 15:23:47,813 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33709-0x101c5bd87ea0033, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-12 15:23:47,813 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-12 15:23:47,814 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:47,815 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-12 15:23:47,815 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase5.apache.org,43095,1689175427762 from backup master directory 2023-07-12 15:23:47,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-12 15:23:47,817 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase5.apache.org,43095,1689175427762 2023-07-12 15:23:47,817 WARN [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:23:47,817 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-12 15:23:47,817 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase5.apache.org,43095,1689175427762 2023-07-12 15:23:47,828 INFO [master/jenkins-hbase5:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:23:47,852 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x3b7e71fc to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:47,855 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6462ba40, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:47,855 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-12 15:23:47,855 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-12 15:23:47,856 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:47,861 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,39489,1689175392691 to hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,39489,1689175392691-dead as it is dead 2023-07-12 15:23:47,862 INFO [master/jenkins-hbase5:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,39489,1689175392691-dead/jenkins-hbase5.apache.org%2C39489%2C1689175392691.1689175393209 2023-07-12 15:23:47,862 INFO [master/jenkins-hbase5:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,39489,1689175392691-dead/jenkins-hbase5.apache.org%2C39489%2C1689175392691.1689175393209 after 0ms 2023-07-12 15:23:47,863 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,39489,1689175392691-dead/jenkins-hbase5.apache.org%2C39489%2C1689175392691.1689175393209 to hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase5.apache.org%2C39489%2C1689175392691.1689175393209 2023-07-12 15:23:47,863 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,39489,1689175392691-dead 2023-07-12 15:23:47,863 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,43095,1689175427762 2023-07-12 15:23:47,865 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C43095%2C1689175427762, suffix=, logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,43095,1689175427762, archiveDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/oldWALs, maxLogs=10 2023-07-12 15:23:47,880 DEBUG [RS-EventLoopGroup-17-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK] 2023-07-12 15:23:47,880 DEBUG [RS-EventLoopGroup-17-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK] 2023-07-12 15:23:47,880 DEBUG [RS-EventLoopGroup-17-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46687,DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4,DISK] 2023-07-12 15:23:47,882 INFO [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/WALs/jenkins-hbase5.apache.org,43095,1689175427762/jenkins-hbase5.apache.org%2C43095%2C1689175427762.1689175427865 2023-07-12 15:23:47,883 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK], DatanodeInfoWithStorage[127.0.0.1:46687,DS-b9fd3bd6-e80b-46e7-9031-5d6f667b56f4,DISK], DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK]] 2023-07-12 15:23:47,883 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:47,883 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:47,883 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:47,883 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:47,887 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:47,888 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-12 15:23:47,888 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-12 15:23:47,894 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/4b68c55f96324a40933fdb475c17211a 2023-07-12 15:23:47,898 INFO [StoreFileOpener-proc-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb4131ab8c434af580fd20da72e7e92a 2023-07-12 15:23:47,898 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/fb4131ab8c434af580fd20da72e7e92a 2023-07-12 15:23:47,898 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:47,899 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-12 15:23:47,899 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase5.apache.org%2C39489%2C1689175392691.1689175393209 2023-07-12 15:23:47,903 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 139, firstSequenceIdInLog=120, maxSequenceIdInLog=240, path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase5.apache.org%2C39489%2C1689175392691.1689175393209 2023-07-12 15:23:47,903 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase5.apache.org%2C39489%2C1689175392691.1689175393209 2023-07-12 15:23:47,906 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:23:47,908 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/240.seqid, newMaxSeqId=240, maxSeqId=118 2023-07-12 15:23:47,909 INFO [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=241; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74004647, jitterRate=0.10275517404079437}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:23:47,909 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:23:47,909 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-12 15:23:47,911 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-12 15:23:47,911 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-12 15:23:47,911 INFO [master/jenkins-hbase5:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:23:47,911 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:23:47,917 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:23:47,918 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:23:47,918 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:23:47,918 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:23:47,918 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-12 15:23:47,919 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-12 15:23:47,919 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-12 15:23:47,919 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-12 15:23:47,920 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:23:47,920 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase5.apache.org,40223,1689175392718, table=hbase:meta, region=1588230740 2023-07-12 15:23:47,922 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 1 possibly 'live' servers, and 0 'splitting'. 2023-07-12 15:23:47,925 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase5.apache.org,40223,1689175392718 already deleted, retry=false 2023-07-12 15:23:47,925 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase5.apache.org,40223,1689175392718 on jenkins-hbase5.apache.org,43095,1689175427762 2023-07-12 15:23:47,926 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase5.apache.org,40223,1689175392718, splitWal=true, meta=true 2023-07-12 15:23:47,926 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=27 for jenkins-hbase5.apache.org,40223,1689175392718 (carryingMeta=true) jenkins-hbase5.apache.org,40223,1689175392718/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@5972f78d[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-12 15:23:47,936 INFO [master/jenkins-hbase5:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-12 15:23:47,936 INFO [master/jenkins-hbase5:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-12 15:23:47,937 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-12 15:23:47,937 INFO [master/jenkins-hbase5:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-12 15:23:47,937 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-12 15:23:47,939 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-12 15:23:47,940 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-12 15:23:47,941 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-12 15:23:47,942 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33709-0x101c5bd87ea0033, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-12 15:23:47,942 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-12 15:23:47,942 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:23:47,942 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase5.apache.org,43095,1689175427762, sessionid=0x101c5bd87ea0032, setting cluster-up flag (Was=false) 2023-07-12 15:23:47,947 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-12 15:23:47,948 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase5.apache.org,43095,1689175427762 2023-07-12 15:23:47,950 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-12 15:23:47,950 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase5.apache.org,43095,1689175427762 2023-07-12 15:23:47,951 WARN [master/jenkins-hbase5:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/.hbase-snapshot/.tmp 2023-07-12 15:23:47,953 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:47,953 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:47,953 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:47,953 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:23:47,953 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase5:0, corePoolSize=10, maxPoolSize=10 2023-07-12 15:23:47,954 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:47,954 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:23:47,954 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:47,956 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689175457956 2023-07-12 15:23:47,957 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-12 15:23:47,957 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-12 15:23:47,957 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-12 15:23:47,957 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-12 15:23:47,957 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-12 15:23:47,957 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-12 15:23:47,957 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:47,958 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase5.apache.org,40223,1689175392718; numProcessing=1 2023-07-12 15:23:47,958 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=27, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,40223,1689175392718, splitWal=true, meta=true 2023-07-12 15:23:47,959 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-12 15:23:47,959 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-12 15:23:47,959 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-12 15:23:47,959 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,40223,1689175392718, splitWal=true, meta=true, isMeta: true 2023-07-12 15:23:47,964 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-12 15:23:47,964 INFO [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-12 15:23:47,964 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175427964,5,FailOnTimeoutGroup] 2023-07-12 15:23:47,964 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175427964,5,FailOnTimeoutGroup] 2023-07-12 15:23:47,964 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:47,964 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-12 15:23:47,964 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:47,964 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:47,964 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689175427964, completionTime=-1 2023-07-12 15:23:47,964 WARN [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-12 15:23:47,964 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-12 15:23:47,965 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,40223,1689175392718-splitting 2023-07-12 15:23:47,966 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,40223,1689175392718-splitting dir is empty, no logs to split. 2023-07-12 15:23:47,966 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase5.apache.org,40223,1689175392718 WAL count=0, meta=true 2023-07-12 15:23:47,968 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,40223,1689175392718-splitting dir is empty, no logs to split. 2023-07-12 15:23:47,968 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase5.apache.org,40223,1689175392718 WAL count=0, meta=true 2023-07-12 15:23:47,968 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase5.apache.org,40223,1689175392718 WAL splitting is done? wals=0, meta=true 2023-07-12 15:23:47,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-12 15:23:47,970 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-12 15:23:47,971 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-12 15:23:48,010 INFO [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(951): ClusterId : 5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:23:48,010 DEBUG [RS:0;jenkins-hbase5:33709] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 15:23:48,012 DEBUG [RS:0;jenkins-hbase5:33709] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 15:23:48,013 DEBUG [RS:0;jenkins-hbase5:33709] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 15:23:48,014 DEBUG [RS:0;jenkins-hbase5:33709] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 15:23:48,016 DEBUG [RS:0;jenkins-hbase5:33709] zookeeper.ReadOnlyZKClient(139): Connect 0x2ae5bf72 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:48,019 DEBUG [RS:0;jenkins-hbase5:33709] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2422545e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:48,020 DEBUG [RS:0;jenkins-hbase5:33709] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@201dd402, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase5.apache.org/172.31.10.131:0 2023-07-12 15:23:48,028 DEBUG [RS:0;jenkins-hbase5:33709] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase5:33709 2023-07-12 15:23:48,028 INFO [RS:0;jenkins-hbase5:33709] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 15:23:48,029 INFO [RS:0;jenkins-hbase5:33709] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 15:23:48,029 DEBUG [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 15:23:48,029 INFO [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase5.apache.org,43095,1689175427762 with isa=jenkins-hbase5.apache.org/172.31.10.131:33709, startcode=1689175427795 2023-07-12 15:23:48,029 DEBUG [RS:0;jenkins-hbase5:33709] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 15:23:48,031 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:32817, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 15:23:48,032 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43095] master.ServerManager(394): Registering regionserver=jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:23:48,032 DEBUG [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143 2023-07-12 15:23:48,032 DEBUG [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:36223 2023-07-12 15:23:48,033 DEBUG [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 15:23:48,034 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101c5bd87ea0026, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:48,034 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:23:48,034 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-12 15:23:48,034 DEBUG [RS:0;jenkins-hbase5:33709] zookeeper.ZKUtil(162): regionserver:33709-0x101c5bd87ea0033, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:23:48,034 WARN [RS:0;jenkins-hbase5:33709] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:23:48,035 INFO [RS:0;jenkins-hbase5:33709] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:48,035 DEBUG [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(1948): logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:23:48,035 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase5.apache.org,33709,1689175427795] 2023-07-12 15:23:48,041 DEBUG [RS:0;jenkins-hbase5:33709] zookeeper.ZKUtil(162): regionserver:33709-0x101c5bd87ea0033, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:23:48,041 DEBUG [RS:0;jenkins-hbase5:33709] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-12 15:23:48,041 INFO [RS:0;jenkins-hbase5:33709] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 15:23:48,043 INFO [RS:0;jenkins-hbase5:33709] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 15:23:48,043 INFO [RS:0;jenkins-hbase5:33709] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 15:23:48,043 INFO [RS:0;jenkins-hbase5:33709] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:48,043 INFO [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 15:23:48,045 INFO [RS:0;jenkins-hbase5:33709] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:48,045 DEBUG [RS:0;jenkins-hbase5:33709] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:48,045 DEBUG [RS:0;jenkins-hbase5:33709] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:48,046 DEBUG [RS:0;jenkins-hbase5:33709] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:48,046 DEBUG [RS:0;jenkins-hbase5:33709] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:48,046 DEBUG [RS:0;jenkins-hbase5:33709] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:48,046 DEBUG [RS:0;jenkins-hbase5:33709] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase5:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:23:48,046 DEBUG [RS:0;jenkins-hbase5:33709] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:48,046 DEBUG [RS:0;jenkins-hbase5:33709] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:48,046 DEBUG [RS:0;jenkins-hbase5:33709] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:48,046 DEBUG [RS:0;jenkins-hbase5:33709] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase5:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:23:48,048 INFO [RS:0;jenkins-hbase5:33709] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:48,048 INFO [RS:0;jenkins-hbase5:33709] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:48,049 INFO [RS:0;jenkins-hbase5:33709] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:48,093 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=129ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-12 15:23:48,095 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1ffdd212 to 127.0.0.1:64130 2023-07-12 15:23:48,095 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:48,095 INFO [RS:0;jenkins-hbase5:33709] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 15:23:48,095 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0fe1f1bf to 127.0.0.1:64130 2023-07-12 15:23:48,096 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:48,096 INFO [RS:0;jenkins-hbase5:33709] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,33709,1689175427795-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:48,107 INFO [RS:0;jenkins-hbase5:33709] regionserver.Replication(203): jenkins-hbase5.apache.org,33709,1689175427795 started 2023-07-12 15:23:48,107 INFO [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(1637): Serving as jenkins-hbase5.apache.org,33709,1689175427795, RpcServer on jenkins-hbase5.apache.org/172.31.10.131:33709, sessionid=0x101c5bd87ea0033 2023-07-12 15:23:48,107 DEBUG [RS:0;jenkins-hbase5:33709] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 15:23:48,107 DEBUG [RS:0;jenkins-hbase5:33709] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:23:48,107 DEBUG [RS:0;jenkins-hbase5:33709] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,33709,1689175427795' 2023-07-12 15:23:48,107 DEBUG [RS:0;jenkins-hbase5:33709] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-12 15:23:48,108 DEBUG [RS:0;jenkins-hbase5:33709] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-12 15:23:48,108 DEBUG [RS:0;jenkins-hbase5:33709] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 15:23:48,108 DEBUG [RS:0;jenkins-hbase5:33709] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 15:23:48,108 DEBUG [RS:0;jenkins-hbase5:33709] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:23:48,108 DEBUG [RS:0;jenkins-hbase5:33709] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase5.apache.org,33709,1689175427795' 2023-07-12 15:23:48,108 DEBUG [RS:0;jenkins-hbase5:33709] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-12 15:23:48,108 DEBUG [RS:0;jenkins-hbase5:33709] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-12 15:23:48,108 DEBUG [RS:0;jenkins-hbase5:33709] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 15:23:48,108 INFO [RS:0;jenkins-hbase5:33709] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 15:23:48,109 INFO [RS:0;jenkins-hbase5:33709] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 15:23:48,121 DEBUG [jenkins-hbase5:43095] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-12 15:23:48,122 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase5.apache.org,33709,1689175427795, state=OPENING 2023-07-12 15:23:48,124 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-12 15:23:48,124 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-12 15:23:48,124 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase5.apache.org,33709,1689175427795}] 2023-07-12 15:23:48,210 INFO [RS:0;jenkins-hbase5:33709] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C33709%2C1689175427795, suffix=, logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33709,1689175427795, archiveDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs, maxLogs=10 2023-07-12 15:23:48,225 DEBUG [RS-EventLoopGroup-17-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK] 2023-07-12 15:23:48,225 DEBUG [RS-EventLoopGroup-17-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK] 2023-07-12 15:23:48,226 DEBUG [RS-EventLoopGroup-17-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK] 2023-07-12 15:23:48,227 INFO [RS:0;jenkins-hbase5:33709] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33709,1689175427795/jenkins-hbase5.apache.org%2C33709%2C1689175427795.1689175428211 2023-07-12 15:23:48,228 DEBUG [RS:0;jenkins-hbase5:33709] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK], DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK], DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK]] 2023-07-12 15:23:48,279 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:23:48,279 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:23:48,281 INFO [RS-EventLoopGroup-17-1] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:48742, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 15:23:48,284 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-12 15:23:48,284 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:48,286 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase5.apache.org%2C33709%2C1689175427795.meta, suffix=.meta, logDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33709,1689175427795, archiveDir=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs, maxLogs=10 2023-07-12 15:23:48,299 DEBUG [RS-EventLoopGroup-17-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK] 2023-07-12 15:23:48,299 DEBUG [RS-EventLoopGroup-17-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK] 2023-07-12 15:23:48,299 DEBUG [RS-EventLoopGroup-17-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK] 2023-07-12 15:23:48,301 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,33709,1689175427795/jenkins-hbase5.apache.org%2C33709%2C1689175427795.meta.1689175428286.meta 2023-07-12 15:23:48,301 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40757,DS-70830774-8243-47cc-a229-fa9f5cfd830d,DISK], DatanodeInfoWithStorage[127.0.0.1:40607,DS-222af625-1417-4c2e-a1f3-46a25780170d,DISK], DatanodeInfoWithStorage[127.0.0.1:38115,DS-f5196444-584a-4f4f-bc5b-b784f48827de,DISK]] 2023-07-12 15:23:48,301 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:48,302 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-12 15:23:48,302 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-12 15:23:48,302 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-12 15:23:48,302 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-12 15:23:48,302 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:48,302 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-12 15:23:48,302 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-12 15:23:48,304 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-12 15:23:48,305 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info 2023-07-12 15:23:48,305 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info 2023-07-12 15:23:48,305 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-12 15:23:48,313 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info/2ba3e980bcde4b55b955f1761e409ce7 2023-07-12 15:23:48,317 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info/f9013e31cdf14f9ca5ad1cff1f13869f 2023-07-12 15:23:48,317 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:48,317 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-12 15:23:48,318 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:23:48,318 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:23:48,318 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-12 15:23:48,319 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:48,319 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-12 15:23:48,320 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/table 2023-07-12 15:23:48,320 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/table 2023-07-12 15:23:48,320 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-12 15:23:48,325 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/table/351115f8a9734cc98bccaabf45019170 2023-07-12 15:23:48,325 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:48,325 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740 2023-07-12 15:23:48,326 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740 2023-07-12 15:23:48,328 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-12 15:23:48,330 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-12 15:23:48,330 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=35; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67822980, jitterRate=0.010641157627105713}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-12 15:23:48,330 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-12 15:23:48,331 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=29, masterSystemTime=1689175428279 2023-07-12 15:23:48,336 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-12 15:23:48,336 INFO [RS_OPEN_META-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-12 15:23:48,337 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase5.apache.org,33709,1689175427795, state=OPEN 2023-07-12 15:23:48,338 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-12 15:23:48,339 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-12 15:23:48,340 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-07-12 15:23:48,340 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase5.apache.org,33709,1689175427795 in 215 msec 2023-07-12 15:23:48,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-07-12 15:23:48,343 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 372 msec 2023-07-12 15:23:49,595 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1631ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-07-12 15:23:50,467 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-12 15:23:51,098 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3134ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-12 15:23:52,500 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4536ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-12 15:23:52,500 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:23:52,504 DEBUG [hconnection-0x7f47a61e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:52,505 INFO [RS-EventLoopGroup-17-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:48744, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:52,513 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=e7e73f29c14aa26ac2fc8049609eeb2d, regionState=OPEN, lastHost=jenkins-hbase5.apache.org,40223,1689175392718, regionLocation=jenkins-hbase5.apache.org,40223,1689175392718, openSeqNum=10 2023-07-12 15:23:52,514 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=19659d0923548261983bde115dd510a9, regionState=OPEN, lastHost=jenkins-hbase5.apache.org,40223,1689175392718, regionLocation=jenkins-hbase5.apache.org,40223,1689175392718, openSeqNum=11 2023-07-12 15:23:52,514 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=144fbe8ee3e556340eac4ab1f11a0604, regionState=OPEN, lastHost=jenkins-hbase5.apache.org,40223,1689175392718, regionLocation=jenkins-hbase5.apache.org,40223,1689175392718, openSeqNum=11 2023-07-12 15:23:52,514 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-12 15:23:52,514 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689175492514 2023-07-12 15:23:52,514 INFO [master/jenkins-hbase5:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689175552514 2023-07-12 15:23:52,514 INFO [master/jenkins-hbase5:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 13 msec 2023-07-12 15:23:52,535 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase5.apache.org,40223,1689175392718 had 4 regions 2023-07-12 15:23:52,535 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,43095,1689175427762-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:52,536 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,43095,1689175427762-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:52,536 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,43095,1689175427762-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:52,536 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase5:43095, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:52,536 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-12 15:23:52,536 WARN [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. is NOT online; state={e7e73f29c14aa26ac2fc8049609eeb2d state=OPEN, ts=1689175432513, server=jenkins-hbase5.apache.org,40223,1689175392718}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-12 15:23:52,537 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase5.apache.org,40223,1689175392718, splitWal=true, meta=true, isMeta: false 2023-07-12 15:23:52,540 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,40223,1689175392718-splitting dir is empty, no logs to split. 2023-07-12 15:23:52,540 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase5.apache.org,40223,1689175392718 WAL count=0, meta=false 2023-07-12 15:23:52,540 WARN [master/jenkins-hbase5:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase5.apache.org,40223,1689175392718/hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d., unknown_server=jenkins-hbase5.apache.org,40223,1689175392718/t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9., unknown_server=jenkins-hbase5.apache.org,40223,1689175392718/t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:52,542 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/WALs/jenkins-hbase5.apache.org,40223,1689175392718-splitting dir is empty, no logs to split. 2023-07-12 15:23:52,542 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase5.apache.org,40223,1689175392718 WAL count=0, meta=false 2023-07-12 15:23:52,542 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase5.apache.org,40223,1689175392718 WAL splitting is done? wals=0, meta=false 2023-07-12 15:23:52,547 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e7e73f29c14aa26ac2fc8049609eeb2d, ASSIGN}, {pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=19659d0923548261983bde115dd510a9, ASSIGN}, {pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=144fbe8ee3e556340eac4ab1f11a0604, ASSIGN}] 2023-07-12 15:23:52,547 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=19659d0923548261983bde115dd510a9, ASSIGN 2023-07-12 15:23:52,548 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e7e73f29c14aa26ac2fc8049609eeb2d, ASSIGN 2023-07-12 15:23:52,548 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=144fbe8ee3e556340eac4ab1f11a0604, ASSIGN 2023-07-12 15:23:52,548 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=19659d0923548261983bde115dd510a9, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-12 15:23:52,549 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=144fbe8ee3e556340eac4ab1f11a0604, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-12 15:23:52,549 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=e7e73f29c14aa26ac2fc8049609eeb2d, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-12 15:23:52,549 DEBUG [jenkins-hbase5:43095] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-12 15:23:52,552 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=19659d0923548261983bde115dd510a9, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:23:52,553 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=144fbe8ee3e556340eac4ab1f11a0604, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:23:52,553 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175432552"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175432552"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175432552"}]},"ts":"1689175432552"} 2023-07-12 15:23:52,553 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=e7e73f29c14aa26ac2fc8049609eeb2d, regionState=OPENING, regionLocation=jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:23:52,553 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175432553"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175432553"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175432553"}]},"ts":"1689175432553"} 2023-07-12 15:23:52,553 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689175432553"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689175432553"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689175432553"}]},"ts":"1689175432553"} 2023-07-12 15:23:52,555 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 19659d0923548261983bde115dd510a9, server=jenkins-hbase5.apache.org,33709,1689175427795}] 2023-07-12 15:23:52,556 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 144fbe8ee3e556340eac4ab1f11a0604, server=jenkins-hbase5.apache.org,33709,1689175427795}] 2023-07-12 15:23:52,559 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=35, ppid=30, state=RUNNABLE; OpenRegionProcedure e7e73f29c14aa26ac2fc8049609eeb2d, server=jenkins-hbase5.apache.org,33709,1689175427795}] 2023-07-12 15:23:52,713 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:52,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e7e73f29c14aa26ac2fc8049609eeb2d, NAME => 'hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:52,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:52,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:52,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:52,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:52,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:52,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 19659d0923548261983bde115dd510a9, NAME => 't1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:52,716 INFO [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:52,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 19659d0923548261983bde115dd510a9 2023-07-12 15:23:52,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:52,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 19659d0923548261983bde115dd510a9 2023-07-12 15:23:52,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 19659d0923548261983bde115dd510a9 2023-07-12 15:23:52,717 DEBUG [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/info 2023-07-12 15:23:52,717 DEBUG [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/info 2023-07-12 15:23:52,718 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 19659d0923548261983bde115dd510a9 2023-07-12 15:23:52,718 INFO [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e7e73f29c14aa26ac2fc8049609eeb2d columnFamilyName info 2023-07-12 15:23:52,719 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1 2023-07-12 15:23:52,719 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1 2023-07-12 15:23:52,719 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 19659d0923548261983bde115dd510a9 columnFamilyName cf1 2023-07-12 15:23:52,723 DEBUG [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/info/98bda8b543444aaf919145966712fca0 2023-07-12 15:23:52,724 INFO [StoreOpener-e7e73f29c14aa26ac2fc8049609eeb2d-1] regionserver.HStore(310): Store=e7e73f29c14aa26ac2fc8049609eeb2d/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:52,724 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:52,725 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:52,728 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:23:52,729 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened e7e73f29c14aa26ac2fc8049609eeb2d; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64239966, jitterRate=-0.04274991154670715}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:23:52,729 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1/82858d2ae5f44600b94c571e38fb835c 2023-07-12 15:23:52,729 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for e7e73f29c14aa26ac2fc8049609eeb2d: 2023-07-12 15:23:52,730 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d., pid=35, masterSystemTime=1689175432709 2023-07-12 15:23:52,733 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:52,733 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:23:52,734 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=e7e73f29c14aa26ac2fc8049609eeb2d, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:23:52,734 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689175432734"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175432734"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175432734"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175432734"}]},"ts":"1689175432734"} 2023-07-12 15:23:52,736 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5c8c7910355405c8d6fb1542e12af8f 2023-07-12 15:23:52,736 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1/d5c8c7910355405c8d6fb1542e12af8f 2023-07-12 15:23:52,736 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(310): Store=19659d0923548261983bde115dd510a9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:52,736 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 19659d0923548261983bde115dd510a9 2023-07-12 15:23:52,737 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=35, resume processing ppid=30 2023-07-12 15:23:52,737 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=30, state=SUCCESS; OpenRegionProcedure e7e73f29c14aa26ac2fc8049609eeb2d, server=jenkins-hbase5.apache.org,33709,1689175427795 in 179 msec 2023-07-12 15:23:52,737 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/norep 2023-07-12 15:23:52,737 DEBUG [StoreOpener-19659d0923548261983bde115dd510a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/norep 2023-07-12 15:23:52,738 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 19659d0923548261983bde115dd510a9 columnFamilyName norep 2023-07-12 15:23:52,738 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=e7e73f29c14aa26ac2fc8049609eeb2d, ASSIGN in 193 msec 2023-07-12 15:23:52,738 INFO [StoreOpener-19659d0923548261983bde115dd510a9-1] regionserver.HStore(310): Store=19659d0923548261983bde115dd510a9/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:52,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9 2023-07-12 15:23:52,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9 2023-07-12 15:23:52,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:23:52,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 19659d0923548261983bde115dd510a9 2023-07-12 15:23:52,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 19659d0923548261983bde115dd510a9; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72268114, jitterRate=0.07687881588935852}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:23:52,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 19659d0923548261983bde115dd510a9: 2023-07-12 15:23:52,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9., pid=33, masterSystemTime=1689175432709 2023-07-12 15:23:52,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:52,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:23:52,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:52,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 144fbe8ee3e556340eac4ab1f11a0604, NAME => 't2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:23:52,747 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=19659d0923548261983bde115dd510a9, regionState=OPEN, openSeqNum=18, regionLocation=jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:23:52,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:52,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:23:52,747 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175432746"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175432746"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175432746"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175432746"}]},"ts":"1689175432746"} 2023-07-12 15:23:52,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7894): checking encryption for 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:52,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(7897): checking classloading for 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:52,748 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:52,749 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1 2023-07-12 15:23:52,749 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1 2023-07-12 15:23:52,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-07-12 15:23:52,750 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 19659d0923548261983bde115dd510a9, server=jenkins-hbase5.apache.org,33709,1689175427795 in 193 msec 2023-07-12 15:23:52,750 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 144fbe8ee3e556340eac4ab1f11a0604 columnFamilyName cf1 2023-07-12 15:23:52,751 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=19659d0923548261983bde115dd510a9, ASSIGN in 206 msec 2023-07-12 15:23:52,755 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4998f64c27364ed09f8f0dc657ca76fe 2023-07-12 15:23:52,755 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1/4998f64c27364ed09f8f0dc657ca76fe 2023-07-12 15:23:52,759 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(539): loaded hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1/90e6bd84cbb748218903eb87a1f792f7 2023-07-12 15:23:52,760 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(310): Store=144fbe8ee3e556340eac4ab1f11a0604/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:52,760 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:52,761 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/norep 2023-07-12 15:23:52,761 DEBUG [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/norep 2023-07-12 15:23:52,761 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 144fbe8ee3e556340eac4ab1f11a0604 columnFamilyName norep 2023-07-12 15:23:52,761 INFO [StoreOpener-144fbe8ee3e556340eac4ab1f11a0604-1] regionserver.HStore(310): Store=144fbe8ee3e556340eac4ab1f11a0604/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:23:52,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:52,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:52,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:23:52,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1055): writing seq id for 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:23:52,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1072): Opened 144fbe8ee3e556340eac4ab1f11a0604; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72970565, jitterRate=0.087346151471138}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:23:52,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(965): Region open journal for 144fbe8ee3e556340eac4ab1f11a0604: 2023-07-12 15:23:52,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604., pid=34, masterSystemTime=1689175432709 2023-07-12 15:23:52,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:52,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase5:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:23:52,769 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=144fbe8ee3e556340eac4ab1f11a0604, regionState=OPEN, openSeqNum=18, regionLocation=jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:23:52,769 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689175432769"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689175432769"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689175432769"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689175432769"}]},"ts":"1689175432769"} 2023-07-12 15:23:52,772 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-07-12 15:23:52,772 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 144fbe8ee3e556340eac4ab1f11a0604, server=jenkins-hbase5.apache.org,33709,1689175427795 in 214 msec 2023-07-12 15:23:52,774 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=27 2023-07-12 15:23:52,774 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase5.apache.org,40223,1689175392718 after splitting done 2023-07-12 15:23:52,774 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=144fbe8ee3e556340eac4ab1f11a0604, ASSIGN in 228 msec 2023-07-12 15:23:52,774 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase5.apache.org,40223,1689175392718 from processing; numProcessing=0 2023-07-12 15:23:52,775 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; ServerCrashProcedure jenkins-hbase5.apache.org,40223,1689175392718, splitWal=true, meta=true in 4.8490 sec 2023-07-12 15:23:53,107 INFO [jenkins-hbase5:33709Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:23:53,537 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-12 15:23:53,550 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-12 15:23:53,556 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-12 15:23:53,557 INFO [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.740sec 2023-07-12 15:23:53,557 INFO [master/jenkins-hbase5:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-12 15:23:53,557 INFO [master/jenkins-hbase5:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-12 15:23:53,557 INFO [master/jenkins-hbase5:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-12 15:23:53,557 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,43095,1689175427762-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-12 15:23:53,557 INFO [master/jenkins-hbase5:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase5.apache.org,43095,1689175427762-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-12 15:23:53,560 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-12 15:23:53,613 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(139): Connect 0x6d75fc75 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:53,618 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@69f8714c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:53,619 DEBUG [hconnection-0x541dd40e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:53,620 INFO [RS-EventLoopGroup-17-1] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:48750, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:53,623 INFO [Listener at localhost/42735] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-12 15:23:53,623 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6d75fc75 to 127.0.0.1:64130 2023-07-12 15:23:53,623 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:23:53,625 INFO [Listener at localhost/42735] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase5.apache.org:43095 after: jenkins-hbase5.apache.org:43095 2023-07-12 15:23:53,625 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(139): Connect 0x06b26b48 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:53,629 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3c357234, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:54,042 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-12 15:23:54,042 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-12 15:23:54,043 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-12 15:23:54,043 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-12 15:23:54,131 DEBUG [hconnection-0x24007aee-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:23:54,132 INFO [RS-EventLoopGroup-17-3] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:48758, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:23:54,155 INFO [Listener at localhost/42735] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689175434155 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:54,158 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): syncupReplication16891754341550x0, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:54,159 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689175434155-0x101c5bd87ea0038 connected Start Replication Server start 2023-07-12 15:23:54,161 INFO [Listener at localhost/42735] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:23:54,173 INFO [Listener at localhost/42735] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-12 15:23:54,174 DEBUG [Listener at localhost/42735] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-12 15:23:54,184 INFO [Listener at localhost/42735] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-12 15:23:54,186 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase5.apache.org,43385,1689175419880] other RSs: [] 2023-07-12 15:23:54,205 DEBUG [Listener at localhost/42735.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x0aac3617 to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:23:54,209 DEBUG [Listener at localhost/42735.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@38e340ae, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:23:54,210 INFO [Listener at localhost/42735.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:23:54,211 DEBUG [Listener at localhost/42735.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:23:54,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101c5bd87ea003a connected 2023-07-12 15:23:54,213 INFO [Listener at localhost/42735.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=098cb589-347a-4654-bf7c-f2a1c06d1dde to cluster=5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:23:57,987 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase5.apache.org,43385,1689175419880/1's WALs to 1689175434161.syncuptool.replication.org,1234,1 2023-07-12 15:23:57,990 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase5.apache.org%2C43385%2C1689175419880.1689175420297 with data PBUF\x08\x8A\x09 2023-07-12 15:23:57,992 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase5.apache.org,43385,1689175419880/1's WALs to 1689175434161.syncuptool.replication.org,1234,1 2023-07-12 15:23:58,107 INFO [jenkins-hbase5:33709Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:23:58,217 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-12 15:23:59,994 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase5.apache.org,43385,1689175419880] 2023-07-12 15:23:59,994 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase5.apache.org,43385,1689175419880] 2023-07-12 15:24:00,006 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase5.apache.org,43385,1689175419880] 2023-07-12 15:24:00,006 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase5.apache.org,43385,1689175419880, ReplicationSource: 1, currentBandwidth=0 2023-07-12 15:24:00,006 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:24:00,029 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880] zookeeper.ReadOnlyZKClient(139): Connect 0x0039fd2f to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:24:00,034 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@3afc824, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:24:00,034 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:64130 2023-07-12 15:24:00,039 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:24:00,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x101c5bd87ea003c connected 2023-07-12 15:24:00,041 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase5.apache.org,43385,1689175419880 (queues=1) is replicating from cluster=098cb589-347a-4654-bf7c-f2a1c06d1dde to cluster=5841a97d-6c8e-4b36-998c-2718e4ded579 2023-07-12 15:24:00,041 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase5.apache.org%2C43385%2C1689175419880 2023-07-12 15:24:00,045 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase5.apache.org,43385,1689175419880, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-12 15:24:00,046 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1-jenkins-hbase5.apache.org,43385,1689175419880] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase5.apache.org%2C43385%2C1689175419880 2023-07-12 15:24:00,057 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1-jenkins-hbase5.apache.org,43385,1689175419880] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs/jenkins-hbase5.apache.org%2C43385%2C1689175419880.1689175420297 2023-07-12 15:24:00,057 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1-jenkins-hbase5.apache.org,43385,1689175419880] regionserver.ReplicationSourceWALReader(162): Read 300 WAL entries eligible for replication 2023-07-12 15:24:00,057 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880.replicationSource.wal-reader.jenkins-hbase5.apache.org%2C43385%2C1689175419880,1-jenkins-hbase5.apache.org,43385,1689175419880] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-12 15:24:00,059 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1-jenkins-hbase5.apache.org,43385,1689175419880] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase5.apache.org,43385,1689175419880 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/e2a1d8488c64dcf2f22986998cf96932/112=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/113=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/114=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/115=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/116=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/117=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/118=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/119=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/120=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/121=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/122=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/123=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/124=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/125=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/126=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/127=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/128=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/129=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/130=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/131=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/132=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/133=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/134=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/135=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/136=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/137=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/138=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/139=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/140=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/141=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/142=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/143=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/144=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/145=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/146=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/147=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/148=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/149=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/150=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/151=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/152=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/153=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/154=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/155=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/156=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/157=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/158=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/159=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/160=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/161=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/162=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/163=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/164=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/165=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/166=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/167=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/168=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/169=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/170=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/171=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/172=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/173=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/174=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/175=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/176=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/177=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/178=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/179=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/180=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/181=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/182=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/183=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/184=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/185=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/186=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/187=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/188=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/189=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/190=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/191=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/192=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/193=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/194=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/195=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/196=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/197=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/198=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/199=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/200=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/201=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/202=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/203=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/204=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/205=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/206=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/207=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/208=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/209=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/210=[#edits: 1 = ],208}, {t1_syncup/e2a1d8488c64dcf2f22986998cf96932/211=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/212=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/213=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/214=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/215=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/216=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/217=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/218=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/219=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/220=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/221=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/222=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/223=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/224=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/225=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/226=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/227=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/228=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/229=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/230=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/231=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/232=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/233=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/234=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/235=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/236=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/237=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/238=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/239=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/240=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/241=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/242=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/243=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/244=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/245=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/246=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/247=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/248=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/249=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/250=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/251=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/252=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/253=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/254=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/255=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/256=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/257=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/258=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/259=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/260=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/261=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/262=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/263=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/264=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/265=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/266=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/267=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/268=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/269=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/270=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/271=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/272=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/273=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/274=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/275=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/276=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/277=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/278=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/279=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/280=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/281=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/282=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/283=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/284=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/285=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/286=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/287=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/288=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/289=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/290=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/291=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/292=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/293=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/294=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/295=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/296=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/297=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/298=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/299=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/300=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/301=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/302=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/303=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/304=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/305=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/306=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/307=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/308=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/309=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/310=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/311=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/312=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/313=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/314=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/315=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/316=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/317=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/318=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/319=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/320=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/321=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/322=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/323=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/324=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/325=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/326=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/327=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/328=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/329=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/330=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/331=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/332=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/333=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/334=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/335=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/336=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/337=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/338=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/339=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/340=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/341=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/342=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/343=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/344=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/345=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/346=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/347=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/348=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/349=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/350=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/351=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/352=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/353=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/354=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/355=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/356=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/357=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/358=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/359=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/360=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/361=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/362=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/363=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/364=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/365=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/366=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/367=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/368=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/369=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/370=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/371=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/372=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/373=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/374=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/375=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/376=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/377=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/378=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/379=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/380=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/381=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/382=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/383=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/384=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/385=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/386=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/387=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/388=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/389=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/390=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/391=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/392=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/393=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/394=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/395=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/396=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/397=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/398=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/399=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/400=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/401=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/402=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/403=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/404=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/405=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/406=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/407=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/408=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/409=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/410=[#edits: 1 = ],208}, {t2_syncup/2c7136e581575f5b2280ddc3c686997b/411=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:41947/user/jenkins/test-data/df203e8b-cad6-33c6-64c5-10615b5840c4/oldWALs/jenkins-hbase5.apache.org%2C43385%2C1689175419880.1689175420297, lastWalPosition=35352, nbRowKeys=300, nbHFiles=0, heapSize=62400, lastSeqIds={}, endOfFile=true,usedBufferSize=62400] 2023-07-12 15:24:00,065 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:24:00,069 INFO [RS-EventLoopGroup-17-2] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:54928, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 15:24:00,074 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33709] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-12 15:24:00,075 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33709] zookeeper.ReadOnlyZKClient(139): Connect 0x32800b0f to 127.0.0.1:64130 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-12 15:24:00,079 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33709] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@3c47f32f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:24:00,082 DEBUG [hconnection-0x3878607f-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:24:00,083 INFO [RS-EventLoopGroup-17-3] ipc.ServerRpcConnection(540): Connection from 172.31.10.131:54942, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=ClientService 2023-07-12 15:24:00,109 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33709] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-12 15:24:00,110 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1-jenkins-hbase5.apache.org,43385,1689175419880] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase5.apache.org,43385,1689175419880] 2023-07-12 15:24:00,113 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1-jenkins-hbase5.apache.org,43385,1689175419880] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase5.apache.org%2C43385%2C1689175419880.1689175420297] 2023-07-12 15:24:00,115 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1-jenkins-hbase5.apache.org,43385,1689175419880] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase5.apache.org,43385,1689175419880 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-12 15:24:00,115 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1-jenkins-hbase5.apache.org,43385,1689175419880] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase5.apache.org%2C43385%2C1689175419880 of peer 1-jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:24:00,215 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1-jenkins-hbase5.apache.org,43385,1689175419880] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:24:00,217 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689175434155-0x101c5bd87ea0038, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689175434161.syncuptool.replication.org,1234,1/1-jenkins-hbase5.apache.org,43385,1689175419880 2023-07-12 15:24:00,217 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880.replicationSource.shipperjenkins-hbase5.apache.org%2C43385%2C1689175419880,1-jenkins-hbase5.apache.org,43385,1689175419880] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase5.apache.org,43385,1689175419880 with the following stats: Total replicated edits: 300, current progress: 2023-07-12 15:24:03,107 INFO [jenkins-hbase5:33709Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-12 15:24:03,107 INFO [jenkins-hbase5:33709Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 300 2023-07-12 15:24:04,186 INFO [Listener at localhost/42735] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-12 15:24:04,289 DEBUG [Listener at localhost/42735.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101c5bd87ea003a, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:24:04,289 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0aac3617 to 127.0.0.1:64130 2023-07-12 15:24:04,291 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:24:04,493 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689175434155-0x101c5bd87ea0038, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:24:04,493 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689175434155-0x101c5bd87ea0038, quorum=127.0.0.1:64130, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:24:04,506 INFO [Listener at localhost/42735] replication.TestReplicationSyncUpTool(280): SyncUpAfterPut succeeded at retry = 0 2023-07-12 15:24:04,506 INFO [Listener at localhost/42735] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-12 15:24:04,506 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x06b26b48 to 127.0.0.1:64130 2023-07-12 15:24:04,506 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:24:04,506 DEBUG [Listener at localhost/42735] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-12 15:24:04,506 DEBUG [Listener at localhost/42735] util.JVMClusterUtil(257): Found active master hash=1753563631, stopped=false 2023-07-12 15:24:04,506 INFO [Listener at localhost/42735] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase5.apache.org,43095,1689175427762 2023-07-12 15:24:04,509 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33709-0x101c5bd87ea0033, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-12 15:24:04,509 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-12 15:24:04,509 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:24:04,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33709-0x101c5bd87ea0033, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:24:04,510 INFO [Listener at localhost/42735] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:24:04,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-12 15:24:04,512 DEBUG [Listener at localhost/42735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3b7e71fc to 127.0.0.1:64130 2023-07-12 15:24:04,512 DEBUG [Listener at localhost/42735] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:24:04,512 INFO [Listener at localhost/42735] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase5.apache.org,33709,1689175427795' ***** 2023-07-12 15:24:04,512 INFO [Listener at localhost/42735] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 15:24:04,516 INFO [RS:0;jenkins-hbase5:33709] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 15:24:04,516 INFO [RS:0;jenkins-hbase5:33709] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 15:24:04,516 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 15:24:04,516 INFO [RS:0;jenkins-hbase5:33709] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 15:24:04,518 INFO [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(3305): Received CLOSE for 144fbe8ee3e556340eac4ab1f11a0604 2023-07-12 15:24:04,518 INFO [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(3305): Received CLOSE for 19659d0923548261983bde115dd510a9 2023-07-12 15:24:04,518 INFO [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(3305): Received CLOSE for e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:24:04,518 INFO [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(1144): stopping server jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:24:04,519 DEBUG [RS:0;jenkins-hbase5:33709] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2ae5bf72 to 127.0.0.1:64130 2023-07-12 15:24:04,519 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 144fbe8ee3e556340eac4ab1f11a0604, disabling compactions & flushes 2023-07-12 15:24:04,520 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:24:04,519 DEBUG [RS:0;jenkins-hbase5:33709] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:24:04,520 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:24:04,520 INFO [RS:0;jenkins-hbase5:33709] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 15:24:04,520 INFO [RS:0;jenkins-hbase5:33709] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 15:24:04,520 INFO [RS:0;jenkins-hbase5:33709] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 15:24:04,520 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. after waiting 0 ms 2023-07-12 15:24:04,520 INFO [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-12 15:24:04,520 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:24:04,520 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing 144fbe8ee3e556340eac4ab1f11a0604 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-12 15:24:04,520 INFO [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-12 15:24:04,520 DEBUG [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(1478): Online Regions={144fbe8ee3e556340eac4ab1f11a0604=t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604., 19659d0923548261983bde115dd510a9=t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9., 1588230740=hbase:meta,,1.1588230740, e7e73f29c14aa26ac2fc8049609eeb2d=hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d.} 2023-07-12 15:24:04,532 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-12 15:24:04,532 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-12 15:24:04,532 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-12 15:24:04,532 DEBUG [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(1504): Waiting on 144fbe8ee3e556340eac4ab1f11a0604, 1588230740, 19659d0923548261983bde115dd510a9, e7e73f29c14aa26ac2fc8049609eeb2d 2023-07-12 15:24:04,532 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-12 15:24:04,532 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-12 15:24:04,533 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-12 15:24:04,562 INFO [regionserver/jenkins-hbase5:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:24:04,565 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=43 (bloomFilter=false), to=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/.tmp/info/7647508560274c2eb2226d2d5c56d0a7 2023-07-12 15:24:04,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=21 (bloomFilter=true), to=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/.tmp/cf1/f17ba4e7acaa43c1ad07d073d06cdfeb 2023-07-12 15:24:04,579 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/.tmp/info/7647508560274c2eb2226d2d5c56d0a7 as hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info/7647508560274c2eb2226d2d5c56d0a7 2023-07-12 15:24:04,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/.tmp/cf1/f17ba4e7acaa43c1ad07d073d06cdfeb as hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1/f17ba4e7acaa43c1ad07d073d06cdfeb 2023-07-12 15:24:04,589 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/info/7647508560274c2eb2226d2d5c56d0a7, entries=24, sequenceid=43, filesize=7.4 K 2023-07-12 15:24:04,591 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/cf1/f17ba4e7acaa43c1ad07d073d06cdfeb, entries=200, sequenceid=21, filesize=12.2 K 2023-07-12 15:24:04,592 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 144fbe8ee3e556340eac4ab1f11a0604 in 72ms, sequenceid=21, compaction requested=true 2023-07-12 15:24:04,595 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 62ms, sequenceid=43, compaction requested=true 2023-07-12 15:24:04,609 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t2_syncup/144fbe8ee3e556340eac4ab1f11a0604/recovered.edits/24.seqid, newMaxSeqId=24, maxSeqId=17 2023-07-12 15:24:04,609 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/meta/1588230740/recovered.edits/46.seqid, newMaxSeqId=46, maxSeqId=34 2023-07-12 15:24:04,610 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:24:04,610 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 144fbe8ee3e556340eac4ab1f11a0604: 2023-07-12 15:24:04,611 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-12 15:24:04,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689175388133.144fbe8ee3e556340eac4ab1f11a0604. 2023-07-12 15:24:04,611 INFO [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-12 15:24:04,611 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-12 15:24:04,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing 19659d0923548261983bde115dd510a9, disabling compactions & flushes 2023-07-12 15:24:04,612 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-12 15:24:04,612 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:24:04,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:24:04,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. after waiting 0 ms 2023-07-12 15:24:04,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:24:04,612 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2745): Flushing 19659d0923548261983bde115dd510a9 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-12 15:24:04,626 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=21 (bloomFilter=true), to=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/.tmp/cf1/54e93b0aea6d4b909ad7b2a32429915d 2023-07-12 15:24:04,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/.tmp/cf1/54e93b0aea6d4b909ad7b2a32429915d as hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1/54e93b0aea6d4b909ad7b2a32429915d 2023-07-12 15:24:04,638 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HStore(1080): Added hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/cf1/54e93b0aea6d4b909ad7b2a32429915d, entries=100, sequenceid=21, filesize=8.3 K 2023-07-12 15:24:04,640 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 19659d0923548261983bde115dd510a9 in 28ms, sequenceid=21, compaction requested=true 2023-07-12 15:24:04,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/default/t1_syncup/19659d0923548261983bde115dd510a9/recovered.edits/24.seqid, newMaxSeqId=24, maxSeqId=17 2023-07-12 15:24:04,650 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:24:04,650 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for 19659d0923548261983bde115dd510a9: 2023-07-12 15:24:04,650 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689175387521.19659d0923548261983bde115dd510a9. 2023-07-12 15:24:04,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1604): Closing e7e73f29c14aa26ac2fc8049609eeb2d, disabling compactions & flushes 2023-07-12 15:24:04,651 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:24:04,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:24:04,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. after waiting 0 ms 2023-07-12 15:24:04,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:24:04,658 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/data/hbase/namespace/e7e73f29c14aa26ac2fc8049609eeb2d/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=12 2023-07-12 15:24:04,660 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:24:04,660 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] regionserver.HRegion(1558): Region close journal for e7e73f29c14aa26ac2fc8049609eeb2d: 2023-07-12 15:24:04,660 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase5:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689175385600.e7e73f29c14aa26ac2fc8049609eeb2d. 2023-07-12 15:24:04,732 INFO [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(1170): stopping server jenkins-hbase5.apache.org,33709,1689175427795; all regions closed. 2023-07-12 15:24:04,739 DEBUG [RS:0;jenkins-hbase5:33709] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs 2023-07-12 15:24:04,739 INFO [RS:0;jenkins-hbase5:33709] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase5.apache.org%2C33709%2C1689175427795.meta:.meta(num 1689175428286) 2023-07-12 15:24:04,747 DEBUG [RS:0;jenkins-hbase5:33709] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/oldWALs 2023-07-12 15:24:04,747 INFO [RS:0;jenkins-hbase5:33709] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase5.apache.org%2C33709%2C1689175427795:(num 1689175428211) 2023-07-12 15:24:04,747 DEBUG [RS:0;jenkins-hbase5:33709] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:24:04,747 INFO [RS:0;jenkins-hbase5:33709] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:24:04,747 INFO [RS:0;jenkins-hbase5:33709] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase5:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-12 15:24:04,747 INFO [regionserver/jenkins-hbase5:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:24:04,748 DEBUG [RS:0;jenkins-hbase5:33709] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x32800b0f to 127.0.0.1:64130 2023-07-12 15:24:04,748 DEBUG [RS:0;jenkins-hbase5:33709] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:24:04,749 INFO [RS:0;jenkins-hbase5:33709] ipc.NettyRpcServer(158): Stopping server on /172.31.10.131:33709 2023-07-12 15:24:04,754 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33709-0x101c5bd87ea0033, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase5.apache.org,33709,1689175427795 2023-07-12 15:24:04,754 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101c5bd87ea0026, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:24:04,754 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x101c5bd87ea003c, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:24:04,754 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33709-0x101c5bd87ea0033, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:24:04,754 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-12 15:24:04,754 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-12 15:24:04,754 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-12 15:24:04,755 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase5.apache.org,33709,1689175427795] 2023-07-12 15:24:04,755 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase5.apache.org,33709,1689175427795; numProcessing=1 2023-07-12 15:24:04,856 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33709-0x101c5bd87ea0033, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:24:04,856 INFO [RS:0;jenkins-hbase5:33709] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase5.apache.org,33709,1689175427795; zookeeper connection closed. 2023-07-12 15:24:04,856 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): regionserver:33709-0x101c5bd87ea0033, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:24:04,856 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@a9c442f] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@a9c442f 2023-07-12 15:24:04,856 INFO [Listener at localhost/42735] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-12 15:24:04,857 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase5.apache.org,33709,1689175427795 already deleted, retry=false 2023-07-12 15:24:04,857 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase5.apache.org,33709,1689175427795 expired; onlineServers=0 2023-07-12 15:24:04,857 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase5.apache.org,43095,1689175427762' ***** 2023-07-12 15:24:04,857 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-12 15:24:04,858 DEBUG [M:0;jenkins-hbase5:43095] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6cbad827, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase5.apache.org/172.31.10.131:0 2023-07-12 15:24:04,858 INFO [M:0;jenkins-hbase5:43095] regionserver.HRegionServer(1144): stopping server jenkins-hbase5.apache.org,43095,1689175427762 2023-07-12 15:24:04,858 INFO [M:0;jenkins-hbase5:43095] regionserver.HRegionServer(1170): stopping server jenkins-hbase5.apache.org,43095,1689175427762; all regions closed. 2023-07-12 15:24:04,858 DEBUG [M:0;jenkins-hbase5:43095] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:24:04,858 DEBUG [M:0;jenkins-hbase5:43095] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-12 15:24:04,858 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-12 15:24:04,858 DEBUG [M:0;jenkins-hbase5:43095] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-12 15:24:04,858 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175427964] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.large.0-1689175427964,5,FailOnTimeoutGroup] 2023-07-12 15:24:04,859 INFO [M:0;jenkins-hbase5:43095] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-12 15:24:04,858 DEBUG [master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175427964] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase5:0:becomeActiveMaster-HFileCleaner.small.0-1689175427964,5,FailOnTimeoutGroup] 2023-07-12 15:24:04,860 INFO [M:0;jenkins-hbase5:43095] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-12 15:24:04,861 INFO [M:0;jenkins-hbase5:43095] hbase.ChoreService(369): Chore service for: master/jenkins-hbase5:0 had [] on shutdown 2023-07-12 15:24:04,861 DEBUG [M:0;jenkins-hbase5:43095] master.HMaster(1512): Stopping service threads 2023-07-12 15:24:04,861 INFO [M:0;jenkins-hbase5:43095] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-12 15:24:04,861 ERROR [M:0;jenkins-hbase5:43095] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:24:04,861 INFO [M:0;jenkins-hbase5:43095] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:24:04,861 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-12 15:24:04,863 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-12 15:24:04,863 DEBUG [M:0;jenkins-hbase5:43095] zookeeper.ZKUtil(398): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-12 15:24:04,863 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-12 15:24:04,863 WARN [M:0;jenkins-hbase5:43095] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-12 15:24:04,863 INFO [M:0;jenkins-hbase5:43095] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:24:04,863 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-12 15:24:04,863 INFO [M:0;jenkins-hbase5:43095] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-12 15:24:04,864 DEBUG [M:0;jenkins-hbase5:43095] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-12 15:24:04,864 INFO [M:0;jenkins-hbase5:43095] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:24:04,864 DEBUG [M:0;jenkins-hbase5:43095] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:24:04,864 DEBUG [M:0;jenkins-hbase5:43095] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-12 15:24:04,864 DEBUG [M:0;jenkins-hbase5:43095] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:24:04,864 INFO [M:0;jenkins-hbase5:43095] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=26.42 KB heapSize=32.85 KB 2023-07-12 15:24:04,877 INFO [M:0;jenkins-hbase5:43095] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=26.42 KB at sequenceid=309 (bloomFilter=true), to=hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/791ad0080dec43089de1ad604f2556d0 2023-07-12 15:24:04,884 DEBUG [M:0;jenkins-hbase5:43095] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/791ad0080dec43089de1ad604f2556d0 as hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/791ad0080dec43089de1ad604f2556d0 2023-07-12 15:24:04,889 INFO [M:0;jenkins-hbase5:43095] regionserver.HStore(1080): Added hdfs://localhost:36223/user/jenkins/test-data/2643b752-84ed-101e-abac-3269a3df8143/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/791ad0080dec43089de1ad604f2556d0, entries=9, sequenceid=309, filesize=5.5 K 2023-07-12 15:24:04,890 INFO [M:0;jenkins-hbase5:43095] regionserver.HRegion(2948): Finished flush of dataSize ~26.42 KB/27051, heapSize ~32.84 KB/33624, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 26ms, sequenceid=309, compaction requested=true 2023-07-12 15:24:04,891 INFO [M:0;jenkins-hbase5:43095] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:24:04,891 DEBUG [M:0;jenkins-hbase5:43095] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:24:04,897 INFO [M:0;jenkins-hbase5:43095] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-12 15:24:04,897 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:24:04,898 INFO [M:0;jenkins-hbase5:43095] ipc.NettyRpcServer(158): Stopping server on /172.31.10.131:43095 2023-07-12 15:24:04,901 DEBUG [M:0;jenkins-hbase5:43095] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase5.apache.org,43095,1689175427762 already deleted, retry=false 2023-07-12 15:24:05,011 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:24:05,011 INFO [M:0;jenkins-hbase5:43095] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase5.apache.org,43095,1689175427762; zookeeper connection closed. 2023-07-12 15:24:05,012 DEBUG [Listener at localhost/42735-EventThread] zookeeper.ZKWatcher(600): master:43095-0x101c5bd87ea0032, quorum=127.0.0.1:64130, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:24:05,013 WARN [Listener at localhost/42735] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-12 15:24:05,018 INFO [Listener at localhost/42735] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-12 15:24:05,123 WARN [BP-1605374934-172.31.10.131-1689175383676 heartbeating to localhost/127.0.0.1:36223] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-12 15:24:05,123 WARN [BP-1605374934-172.31.10.131-1689175383676 heartbeating to localhost/127.0.0.1:36223] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1605374934-172.31.10.131-1689175383676 (Datanode Uuid b1a88679-2fde-4cbc-9840-014883a8d08e) service to localhost/127.0.0.1:36223 2023-07-12 15:24:05,125 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/cluster_28b984ac-22b7-2eea-77a9-387577b549fd/dfs/data/data7/current/BP-1605374934-172.31.10.131-1689175383676] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:24:05,125 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/cluster_28b984ac-22b7-2eea-77a9-387577b549fd/dfs/data/data8/current/BP-1605374934-172.31.10.131-1689175383676] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:24:05,127 WARN [Listener at localhost/42735] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-12 15:24:05,130 INFO [Listener at localhost/42735] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-12 15:24:05,233 WARN [BP-1605374934-172.31.10.131-1689175383676 heartbeating to localhost/127.0.0.1:36223] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-12 15:24:05,235 WARN [BP-1605374934-172.31.10.131-1689175383676 heartbeating to localhost/127.0.0.1:36223] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1605374934-172.31.10.131-1689175383676 (Datanode Uuid 6016f2dd-9b70-49ca-bffc-7a8f7192b3d2) service to localhost/127.0.0.1:36223 2023-07-12 15:24:05,236 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/cluster_28b984ac-22b7-2eea-77a9-387577b549fd/dfs/data/data5/current/BP-1605374934-172.31.10.131-1689175383676] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:24:05,236 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/cluster_28b984ac-22b7-2eea-77a9-387577b549fd/dfs/data/data6/current/BP-1605374934-172.31.10.131-1689175383676] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:24:05,238 WARN [Listener at localhost/42735] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-12 15:24:05,240 INFO [Listener at localhost/42735] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-12 15:24:05,344 WARN [BP-1605374934-172.31.10.131-1689175383676 heartbeating to localhost/127.0.0.1:36223] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-12 15:24:05,344 WARN [BP-1605374934-172.31.10.131-1689175383676 heartbeating to localhost/127.0.0.1:36223] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1605374934-172.31.10.131-1689175383676 (Datanode Uuid a7f7fb07-249b-4ff8-88f6-13227ee5a414) service to localhost/127.0.0.1:36223 2023-07-12 15:24:05,345 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/cluster_28b984ac-22b7-2eea-77a9-387577b549fd/dfs/data/data3/current/BP-1605374934-172.31.10.131-1689175383676] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:24:05,345 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/cluster_28b984ac-22b7-2eea-77a9-387577b549fd/dfs/data/data4/current/BP-1605374934-172.31.10.131-1689175383676] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:24:05,348 WARN [Listener at localhost/42735] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-12 15:24:05,353 INFO [Listener at localhost/42735] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-12 15:24:05,457 WARN [BP-1605374934-172.31.10.131-1689175383676 heartbeating to localhost/127.0.0.1:36223] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-12 15:24:05,458 WARN [BP-1605374934-172.31.10.131-1689175383676 heartbeating to localhost/127.0.0.1:36223] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1605374934-172.31.10.131-1689175383676 (Datanode Uuid 2a2a9a00-f087-4236-9d70-c0b5998a3c34) service to localhost/127.0.0.1:36223 2023-07-12 15:24:05,458 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/cluster_28b984ac-22b7-2eea-77a9-387577b549fd/dfs/data/data1/current/BP-1605374934-172.31.10.131-1689175383676] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:24:05,458 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5a62c720-5550-d698-9410-c7b2563b41ed/cluster_28b984ac-22b7-2eea-77a9-387577b549fd/dfs/data/data2/current/BP-1605374934-172.31.10.131-1689175383676] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:24:05,486 INFO [Listener at localhost/42735] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-12 15:24:05,657 INFO [Listener at localhost/42735] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-12 15:24:05,657 INFO [Listener at localhost/42735] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-12 15:24:05,657 WARN [Listener at localhost/42735] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-12 15:24:05,659 INFO [Listener at localhost/42735] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-12 15:24:05,718 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-12 15:24:05,762 WARN [BP-916305998-172.31.10.131-1689175376370 heartbeating to localhost/127.0.0.1:41947] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-12 15:24:05,763 WARN [BP-916305998-172.31.10.131-1689175376370 heartbeating to localhost/127.0.0.1:41947] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-916305998-172.31.10.131-1689175376370 (Datanode Uuid d8ff5e8d-40a1-4929-b4c9-36d95a7996e3) service to localhost/127.0.0.1:41947 2023-07-12 15:24:05,763 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/cluster_fd1ef24c-43dc-8e79-0fe3-8873ef60ba76/dfs/data/data3/current/BP-916305998-172.31.10.131-1689175376370] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:24:05,764 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/cluster_fd1ef24c-43dc-8e79-0fe3-8873ef60ba76/dfs/data/data4/current/BP-916305998-172.31.10.131-1689175376370] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:24:05,764 WARN [Listener at localhost/42735] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-12 15:24:05,766 INFO [Listener at localhost/42735] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-12 15:24:05,868 WARN [BP-916305998-172.31.10.131-1689175376370 heartbeating to localhost/127.0.0.1:41947] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-12 15:24:05,869 WARN [BP-916305998-172.31.10.131-1689175376370 heartbeating to localhost/127.0.0.1:41947] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-916305998-172.31.10.131-1689175376370 (Datanode Uuid ebf16695-f1f5-40c3-8513-9c1fecd94eff) service to localhost/127.0.0.1:41947 2023-07-12 15:24:05,869 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/cluster_fd1ef24c-43dc-8e79-0fe3-8873ef60ba76/dfs/data/data1/current/BP-916305998-172.31.10.131-1689175376370] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:24:05,870 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8685f0ef-51a3-8aeb-3589-2316d12cab2b/cluster_fd1ef24c-43dc-8e79-0fe3-8873ef60ba76/dfs/data/data2/current/BP-916305998-172.31.10.131-1689175376370] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:24:05,878 INFO [Listener at localhost/42735] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-12 15:24:05,993 INFO [Listener at localhost/42735] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-12 15:24:06,024 INFO [Listener at localhost/42735] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-12 15:24:06,045 INFO [Listener at localhost/42735] hbase.ResourceChecker(175): after: replication.TestReplicationSyncUpTool#testSyncUpTool Thread=177 (was 10) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost:36223 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:36223 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873-SendThread(127.0.0.1:64130) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (535977463) connection to localhost/127.0.0.1:41947 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-6-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64130@0x7b98a54a sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/621636720.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64130@0x0039fd2f-SendThread(127.0.0.1:64130) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:41947 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase5:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase5:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-14 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase5:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-17-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64130@0x7b98a54a-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64130@0x7b98a54a-SendThread(127.0.0.1:64130) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64130@0x0039fd2f sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/621636720.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-15 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase5:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Client (535977463) connection to localhost/127.0.0.1:36223 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase5:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost:36223 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@5eb54af4 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64130@0x0039fd2f-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:41947 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64130@0x04eafa70 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/621636720.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-17-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64130@0x04eafa70-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,43385,1689175419880-SendThread(127.0.0.1:64130) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: regionserver/jenkins-hbase5:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase5:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase5:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,36017,1689175380873-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase5.apache.org,35567,1689175380829-SendThread(127.0.0.1:64130) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase5:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-9 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:36223 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost:41947 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:36223 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (535977463) connection to localhost/127.0.0.1:36223 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:36223 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase5:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-10 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-11 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:36223 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-17-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (535977463) connection to localhost/127.0.0.1:41947 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64130@0x04eafa70-SendThread(127.0.0.1:64130) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-16 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-12 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/42735 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-13 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:41947 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase5:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (535977463) connection to localhost/127.0.0.1:36223 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=654 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=225 (was 165) - SystemLoadAverage LEAK? -, ProcessCount=163 (was 165), AvailableMemoryMB=7328 (was 8380)