2023-07-18 04:54:08,556 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0 2023-07-18 04:54:08,564 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/hadoop-log-dir so I do NOT create it in target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219 2023-07-18 04:54:08,564 WARN [main] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/hadoop-log-dir Erasing configuration value by system value. 2023-07-18 04:54:08,565 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/hadoop-tmp-dir so I do NOT create it in target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219 2023-07-18 04:54:08,565 WARN [main] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-18 04:54:08,565 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219 2023-07-18 04:54:08,578 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool timeout: 13 mins 2023-07-18 04:54:08,579 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL timeout: 13 mins 2023-07-18 04:54:08,608 INFO [Time-limited test] hbase.ResourceChecker(147): before: replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL#testSyncUpTool Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=229, ProcessCount=173, AvailableMemoryMB=7044 2023-07-18 04:54:08,620 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/hadoop-log-dir so I do NOT create it in target/test-data/51d72477-9899-c591-daff-3e898c2f8abf 2023-07-18 04:54:08,620 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/hadoop-log-dir Erasing configuration value by system value. 2023-07-18 04:54:08,621 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/hadoop-tmp-dir so I do NOT create it in target/test-data/51d72477-9899-c591-daff-3e898c2f8abf 2023-07-18 04:54:08,621 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-18 04:54:08,621 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/51d72477-9899-c591-daff-3e898c2f8abf 2023-07-18 04:54:08,628 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/hadoop-log-dir so I do NOT create it in target/test-data/28028910-c7a9-312a-de32-83e0b6b2fbaf 2023-07-18 04:54:08,629 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/hadoop-log-dir Erasing configuration value by system value. 2023-07-18 04:54:08,629 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/hadoop-tmp-dir so I do NOT create it in target/test-data/28028910-c7a9-312a-de32-83e0b6b2fbaf 2023-07-18 04:54:08,629 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-18 04:54:08,630 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28028910-c7a9-312a-de32-83e0b6b2fbaf 2023-07-18 04:54:08,633 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/cluster_33cb87ab-e4a2-016e-8b73-ae4d9306366b, deleteOnExit=true 2023-07-18 04:54:08,753 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/cluster_33cb87ab-e4a2-016e-8b73-ae4d9306366b/zookeeper_0, clientPort=63236, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/cluster_33cb87ab-e4a2-016e-8b73-ae4d9306366b/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/cluster_33cb87ab-e4a2-016e-8b73-ae4d9306366b/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-18 04:54:08,769 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=63236 2023-07-18 04:54:08,772 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=2, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-18 04:54:08,773 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-18 04:54:08,773 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/test.cache.data in system properties and HBase conf 2023-07-18 04:54:08,774 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/hadoop.tmp.dir in system properties and HBase conf 2023-07-18 04:54:08,774 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/hadoop.log.dir in system properties and HBase conf 2023-07-18 04:54:08,775 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-18 04:54:08,775 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-18 04:54:08,775 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-18 04:54:08,879 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-18 04:54:09,284 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-18 04:54:09,288 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-18 04:54:09,288 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-18 04:54:09,288 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-18 04:54:09,289 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-18 04:54:09,289 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-18 04:54:09,289 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-18 04:54:09,289 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-18 04:54:09,290 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-18 04:54:09,290 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-18 04:54:09,290 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/nfs.dump.dir in system properties and HBase conf 2023-07-18 04:54:09,290 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/java.io.tmpdir in system properties and HBase conf 2023-07-18 04:54:09,291 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-18 04:54:09,291 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-18 04:54:09,291 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-18 04:54:09,781 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-18 04:54:09,785 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-18 04:54:10,063 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-18 04:54:10,198 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-18 04:54:10,217 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 04:54:10,253 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 04:54:10,287 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/java.io.tmpdir/Jetty_localhost_39909_hdfs____tcdcp2/webapp 2023-07-18 04:54:10,428 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39909 2023-07-18 04:54:10,446 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-18 04:54:10,446 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-18 04:54:10,966 WARN [Listener at localhost/32937] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 04:54:11,034 WARN [Listener at localhost/32937] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-18 04:54:11,060 WARN [Listener at localhost/32937] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 04:54:11,069 INFO [Listener at localhost/32937] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 04:54:11,076 INFO [Listener at localhost/32937] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/java.io.tmpdir/Jetty_localhost_45243_datanode____.r7oxzd/webapp 2023-07-18 04:54:11,190 INFO [Listener at localhost/32937] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45243 2023-07-18 04:54:11,521 WARN [Listener at localhost/36927] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 04:54:11,539 WARN [Listener at localhost/36927] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-18 04:54:11,546 WARN [Listener at localhost/36927] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 04:54:11,549 INFO [Listener at localhost/36927] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 04:54:11,554 INFO [Listener at localhost/36927] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/java.io.tmpdir/Jetty_localhost_43951_datanode____nh9va5/webapp 2023-07-18 04:54:11,652 INFO [Listener at localhost/36927] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43951 2023-07-18 04:54:11,662 WARN [Listener at localhost/46199] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 04:54:11,964 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb7b97d352cf210b: Processing first storage report for DS-15daf992-9329-4b25-9667-a096fb2deda0 from datanode d7b9589a-9b20-4c9b-be4d-dff0a025932b 2023-07-18 04:54:11,966 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb7b97d352cf210b: from storage DS-15daf992-9329-4b25-9667-a096fb2deda0 node DatanodeRegistration(127.0.0.1:34717, datanodeUuid=d7b9589a-9b20-4c9b-be4d-dff0a025932b, infoPort=39835, infoSecurePort=0, ipcPort=36927, storageInfo=lv=-57;cid=testClusterID;nsid=820929443;c=1689656049856), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-07-18 04:54:11,966 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x5ac2f69f5754395e: Processing first storage report for DS-63fc3a07-0ff5-4e01-8767-68d629b125e2 from datanode 1403a88c-ea03-40ca-b217-fac66b980697 2023-07-18 04:54:11,966 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x5ac2f69f5754395e: from storage DS-63fc3a07-0ff5-4e01-8767-68d629b125e2 node DatanodeRegistration(127.0.0.1:36951, datanodeUuid=1403a88c-ea03-40ca-b217-fac66b980697, infoPort=36609, infoSecurePort=0, ipcPort=46199, storageInfo=lv=-57;cid=testClusterID;nsid=820929443;c=1689656049856), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 04:54:11,967 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb7b97d352cf210b: Processing first storage report for DS-37438740-d44a-48c7-bbd7-c81e59f2bd98 from datanode d7b9589a-9b20-4c9b-be4d-dff0a025932b 2023-07-18 04:54:11,967 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb7b97d352cf210b: from storage DS-37438740-d44a-48c7-bbd7-c81e59f2bd98 node DatanodeRegistration(127.0.0.1:34717, datanodeUuid=d7b9589a-9b20-4c9b-be4d-dff0a025932b, infoPort=39835, infoSecurePort=0, ipcPort=36927, storageInfo=lv=-57;cid=testClusterID;nsid=820929443;c=1689656049856), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 04:54:11,967 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x5ac2f69f5754395e: Processing first storage report for DS-67f45d3f-f271-41c3-8a2d-60a0ebe25e5b from datanode 1403a88c-ea03-40ca-b217-fac66b980697 2023-07-18 04:54:11,967 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x5ac2f69f5754395e: from storage DS-67f45d3f-f271-41c3-8a2d-60a0ebe25e5b node DatanodeRegistration(127.0.0.1:36951, datanodeUuid=1403a88c-ea03-40ca-b217-fac66b980697, infoPort=36609, infoSecurePort=0, ipcPort=46199, storageInfo=lv=-57;cid=testClusterID;nsid=820929443;c=1689656049856), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 04:54:12,074 DEBUG [Listener at localhost/46199] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0 2023-07-18 04:54:12,081 INFO [Listener at localhost/46199] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:12,083 INFO [Listener at localhost/46199] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:12,747 INFO [Listener at localhost/46199] util.FSUtils(471): Created version file at hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c with version=8 2023-07-18 04:54:12,747 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/hbase-staging 2023-07-18 04:54:13,085 INFO [Listener at localhost/46199] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-18 04:54:13,567 INFO [Listener at localhost/46199] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-18 04:54:13,598 INFO [Listener at localhost/46199] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:13,598 INFO [Listener at localhost/46199] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:13,598 INFO [Listener at localhost/46199] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 04:54:13,599 INFO [Listener at localhost/46199] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:13,599 INFO [Listener at localhost/46199] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 04:54:13,738 INFO [Listener at localhost/46199] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 04:54:13,809 DEBUG [Listener at localhost/46199] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-18 04:54:13,904 INFO [Listener at localhost/46199] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44113 2023-07-18 04:54:13,914 INFO [Listener at localhost/46199] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:13,917 INFO [Listener at localhost/46199] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:13,938 INFO [Listener at localhost/46199] zookeeper.RecoverableZooKeeper(93): Process identifier=master:44113 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:13,980 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:441130x0, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:13,982 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:44113-0x10176c7bf3c0000 connected 2023-07-18 04:54:14,007 DEBUG [Listener at localhost/46199] zookeeper.ZKUtil(164): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-18 04:54:14,008 DEBUG [Listener at localhost/46199] zookeeper.ZKUtil(164): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 04:54:14,011 DEBUG [Listener at localhost/46199] zookeeper.ZKUtil(164): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-18 04:54:14,019 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44113 2023-07-18 04:54:14,019 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44113 2023-07-18 04:54:14,020 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44113 2023-07-18 04:54:14,020 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44113 2023-07-18 04:54:14,021 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44113 2023-07-18 04:54:14,025 INFO [Listener at localhost/46199] master.HMaster(444): hbase.rootdir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c, hbase.cluster.distributed=false 2023-07-18 04:54:14,097 INFO [Listener at localhost/46199] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-18 04:54:14,097 INFO [Listener at localhost/46199] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:14,098 INFO [Listener at localhost/46199] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:14,098 INFO [Listener at localhost/46199] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 04:54:14,098 INFO [Listener at localhost/46199] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:14,098 INFO [Listener at localhost/46199] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 04:54:14,103 INFO [Listener at localhost/46199] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 04:54:14,107 INFO [Listener at localhost/46199] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42175 2023-07-18 04:54:14,110 INFO [Listener at localhost/46199] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 04:54:14,116 DEBUG [Listener at localhost/46199] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 04:54:14,117 INFO [Listener at localhost/46199] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:14,120 INFO [Listener at localhost/46199] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:14,122 INFO [Listener at localhost/46199] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42175 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:14,125 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:421750x0, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:14,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42175-0x10176c7bf3c0001 connected 2023-07-18 04:54:14,126 DEBUG [Listener at localhost/46199] zookeeper.ZKUtil(164): regionserver:42175-0x10176c7bf3c0001, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-18 04:54:14,128 DEBUG [Listener at localhost/46199] zookeeper.ZKUtil(164): regionserver:42175-0x10176c7bf3c0001, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 04:54:14,129 DEBUG [Listener at localhost/46199] zookeeper.ZKUtil(164): regionserver:42175-0x10176c7bf3c0001, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-18 04:54:14,129 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42175 2023-07-18 04:54:14,129 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42175 2023-07-18 04:54:14,131 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42175 2023-07-18 04:54:14,131 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42175 2023-07-18 04:54:14,131 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42175 2023-07-18 04:54:14,144 INFO [Listener at localhost/46199] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-18 04:54:14,144 INFO [Listener at localhost/46199] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:14,144 INFO [Listener at localhost/46199] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:14,145 INFO [Listener at localhost/46199] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 04:54:14,145 INFO [Listener at localhost/46199] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:14,145 INFO [Listener at localhost/46199] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 04:54:14,145 INFO [Listener at localhost/46199] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 04:54:14,147 INFO [Listener at localhost/46199] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45857 2023-07-18 04:54:14,148 INFO [Listener at localhost/46199] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 04:54:14,149 DEBUG [Listener at localhost/46199] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 04:54:14,150 INFO [Listener at localhost/46199] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:14,152 INFO [Listener at localhost/46199] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:14,154 INFO [Listener at localhost/46199] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45857 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:14,159 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:458570x0, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:14,160 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45857-0x10176c7bf3c0002 connected 2023-07-18 04:54:14,160 DEBUG [Listener at localhost/46199] zookeeper.ZKUtil(164): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-18 04:54:14,161 DEBUG [Listener at localhost/46199] zookeeper.ZKUtil(164): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 04:54:14,161 DEBUG [Listener at localhost/46199] zookeeper.ZKUtil(164): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-18 04:54:14,162 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45857 2023-07-18 04:54:14,162 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45857 2023-07-18 04:54:14,163 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45857 2023-07-18 04:54:14,166 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45857 2023-07-18 04:54:14,166 DEBUG [Listener at localhost/46199] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45857 2023-07-18 04:54:14,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,44113,1689656052898 2023-07-18 04:54:14,184 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-18 04:54:14,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,44113,1689656052898 2023-07-18 04:54:14,204 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-18 04:54:14,204 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:42175-0x10176c7bf3c0001, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-18 04:54:14,204 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-18 04:54:14,204 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:14,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-18 04:54:14,208 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,44113,1689656052898 from backup master directory 2023-07-18 04:54:14,208 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-18 04:54:14,211 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,44113,1689656052898 2023-07-18 04:54:14,211 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-18 04:54:14,212 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 04:54:14,212 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,44113,1689656052898 2023-07-18 04:54:14,214 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-18 04:54:14,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-18 04:54:14,306 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/hbase.id with ID: 4b2ea01c-df7d-4e18-8185-4db3e50f17db 2023-07-18 04:54:14,348 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:14,366 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:14,409 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x16d9e291 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:14,444 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1a00fc27, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:14,468 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 04:54:14,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-18 04:54:14,479 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:14,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:14,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store-tmp 2023-07-18 04:54:14,552 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:14,552 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-18 04:54:14,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:14,552 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:14,552 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-18 04:54:14,552 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:14,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:14,552 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 04:54:14,554 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/WALs/jenkins-hbase4.apache.org,44113,1689656052898 2023-07-18 04:54:14,556 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:14,576 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44113%2C1689656052898.jenkins-hbase4.apache.org%2C44113%2C1689656052898.regiongroup-0, suffix=, logDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/WALs/jenkins-hbase4.apache.org,44113,1689656052898, archiveDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/oldWALs, maxLogs=10 2023-07-18 04:54:14,596 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.CommonFSUtils$DfsBuilderUtility(753): Could not find replicate method on builder; will not set replicate when creating output stream java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DistributedFileSystem$HdfsDataOutputStreamBuilder.replicate() at java.lang.Class.getMethod(Class.java:1786) at org.apache.hadoop.hbase.util.CommonFSUtils$DfsBuilderUtility.(CommonFSUtils.java:750) at org.apache.hadoop.hbase.util.CommonFSUtils.createForWal(CommonFSUtils.java:802) at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.initOutput(ProtobufLogWriter.java:102) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.FSHLogProvider.createWriter(FSHLogProvider.java:78) at org.apache.hadoop.hbase.regionserver.wal.FSHLog.createWriterInstance(FSHLog.java:307) at org.apache.hadoop.hbase.regionserver.wal.FSHLog.createWriterInstance(FSHLog.java:70) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:195) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:215) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-18 04:54:14,622 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/WALs/jenkins-hbase4.apache.org,44113,1689656052898/jenkins-hbase4.apache.org%2C44113%2C1689656052898.jenkins-hbase4.apache.org%2C44113%2C1689656052898.regiongroup-0.1689656054594 2023-07-18 04:54:14,622 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36951,DS-63fc3a07-0ff5-4e01-8767-68d629b125e2,DISK], DatanodeInfoWithStorage[127.0.0.1:34717,DS-15daf992-9329-4b25-9667-a096fb2deda0,DISK]] 2023-07-18 04:54:14,623 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:14,624 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:14,627 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:14,628 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:14,682 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:14,691 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-18 04:54:14,723 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-18 04:54:14,739 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:14,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:14,746 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:14,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:14,768 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 04:54:14,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73042882, jitterRate=0.08842375874519348}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 04:54:14,769 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 04:54:14,770 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-18 04:54:14,789 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-18 04:54:14,789 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-18 04:54:14,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-18 04:54:14,793 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-18 04:54:14,827 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 34 msec 2023-07-18 04:54:14,827 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-18 04:54:14,853 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-18 04:54:14,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-18 04:54:14,885 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-18 04:54:14,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-18 04:54:14,890 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-18 04:54:14,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-18 04:54:14,899 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-18 04:54:14,902 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:14,904 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-18 04:54:14,904 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-18 04:54:14,919 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-18 04:54:14,923 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-18 04:54:14,923 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-18 04:54:14,923 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:42175-0x10176c7bf3c0001, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-18 04:54:14,923 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:14,924 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,44113,1689656052898, sessionid=0x10176c7bf3c0000, setting cluster-up flag (Was=false) 2023-07-18 04:54:14,945 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:14,952 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-18 04:54:14,954 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,44113,1689656052898 2023-07-18 04:54:14,959 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:14,965 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-18 04:54:14,966 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,44113,1689656052898 2023-07-18 04:54:14,968 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.hbase-snapshot/.tmp 2023-07-18 04:54:14,973 INFO [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(951): ClusterId : 4b2ea01c-df7d-4e18-8185-4db3e50f17db 2023-07-18 04:54:14,973 INFO [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(951): ClusterId : 4b2ea01c-df7d-4e18-8185-4db3e50f17db 2023-07-18 04:54:14,978 DEBUG [RS:0;jenkins-hbase4:42175] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 04:54:14,978 DEBUG [RS:1;jenkins-hbase4:45857] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 04:54:14,984 DEBUG [RS:1;jenkins-hbase4:45857] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 04:54:14,984 DEBUG [RS:0;jenkins-hbase4:42175] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 04:54:14,984 DEBUG [RS:1;jenkins-hbase4:45857] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 04:54:14,985 DEBUG [RS:0;jenkins-hbase4:42175] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 04:54:14,988 DEBUG [RS:1;jenkins-hbase4:45857] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 04:54:14,988 DEBUG [RS:0;jenkins-hbase4:42175] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 04:54:14,989 DEBUG [RS:1;jenkins-hbase4:45857] zookeeper.ReadOnlyZKClient(139): Connect 0x7a1a433c to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:14,990 DEBUG [RS:0;jenkins-hbase4:42175] zookeeper.ReadOnlyZKClient(139): Connect 0x18c5f288 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:15,003 DEBUG [RS:1;jenkins-hbase4:45857] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@74ddb20f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:15,003 DEBUG [RS:1;jenkins-hbase4:45857] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@70e2284c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 04:54:15,003 DEBUG [RS:0;jenkins-hbase4:42175] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@16021fe0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:15,004 DEBUG [RS:0;jenkins-hbase4:42175] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@123d99f8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 04:54:15,027 DEBUG [RS:1;jenkins-hbase4:45857] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:45857 2023-07-18 04:54:15,030 INFO [RS:1;jenkins-hbase4:45857] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 04:54:15,030 INFO [RS:1;jenkins-hbase4:45857] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 04:54:15,030 DEBUG [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 04:54:15,033 DEBUG [RS:0;jenkins-hbase4:42175] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:42175 2023-07-18 04:54:15,033 INFO [RS:0;jenkins-hbase4:42175] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 04:54:15,034 INFO [RS:0;jenkins-hbase4:42175] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 04:54:15,034 DEBUG [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 04:54:15,035 INFO [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,44113,1689656052898 with isa=jenkins-hbase4.apache.org/172.31.14.131:42175, startcode=1689656054096 2023-07-18 04:54:15,035 INFO [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,44113,1689656052898 with isa=jenkins-hbase4.apache.org/172.31.14.131:45857, startcode=1689656054144 2023-07-18 04:54:15,053 DEBUG [RS:0;jenkins-hbase4:42175] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 04:54:15,053 DEBUG [RS:1;jenkins-hbase4:45857] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 04:54:15,102 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-18 04:54:15,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:15,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:15,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:15,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:15,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-18 04:54:15,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 04:54:15,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,125 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689656085125 2023-07-18 04:54:15,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-18 04:54:15,133 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-18 04:54:15,134 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-18 04:54:15,141 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-18 04:54:15,145 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-18 04:54:15,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-18 04:54:15,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-18 04:54:15,157 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-18 04:54:15,157 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-18 04:54:15,158 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,159 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-18 04:54:15,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-18 04:54:15,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-18 04:54:15,164 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-18 04:54:15,164 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-18 04:54:15,166 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656055166,5,FailOnTimeoutGroup] 2023-07-18 04:54:15,167 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656055167,5,FailOnTimeoutGroup] 2023-07-18 04:54:15,167 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,167 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-18 04:54:15,168 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,168 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,198 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57165, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 04:54:15,198 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51347, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 04:54:15,202 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-18 04:54:15,203 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-18 04:54:15,204 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c 2023-07-18 04:54:15,213 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44113] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:15,228 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44113] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42175,1689656054096 2023-07-18 04:54:15,236 DEBUG [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c 2023-07-18 04:54:15,237 DEBUG [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:32937 2023-07-18 04:54:15,237 DEBUG [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 04:54:15,236 DEBUG [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c 2023-07-18 04:54:15,237 DEBUG [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:32937 2023-07-18 04:54:15,237 DEBUG [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 04:54:15,240 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:15,243 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 04:54:15,244 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-18 04:54:15,245 DEBUG [RS:1;jenkins-hbase4:45857] zookeeper.ZKUtil(162): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:15,245 WARN [RS:1;jenkins-hbase4:45857] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 04:54:15,246 INFO [RS:1;jenkins-hbase4:45857] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:15,246 INFO [RS:1;jenkins-hbase4:45857] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:15,247 DEBUG [RS:0;jenkins-hbase4:42175] zookeeper.ZKUtil(162): regionserver:42175-0x10176c7bf3c0001, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42175,1689656054096 2023-07-18 04:54:15,247 DEBUG [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(1948): logDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:15,247 WARN [RS:0;jenkins-hbase4:42175] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 04:54:15,247 INFO [RS:0;jenkins-hbase4:42175] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:15,247 INFO [RS:0;jenkins-hbase4:42175] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:15,247 DEBUG [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(1948): logDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096 2023-07-18 04:54:15,247 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42175,1689656054096] 2023-07-18 04:54:15,248 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45857,1689656054144] 2023-07-18 04:54:15,251 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/info 2023-07-18 04:54:15,253 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-18 04:54:15,256 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:15,256 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-18 04:54:15,259 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/rep_barrier 2023-07-18 04:54:15,259 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-18 04:54:15,260 DEBUG [RS:0;jenkins-hbase4:42175] zookeeper.ZKUtil(162): regionserver:42175-0x10176c7bf3c0001, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42175,1689656054096 2023-07-18 04:54:15,260 DEBUG [RS:1;jenkins-hbase4:45857] zookeeper.ZKUtil(162): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,42175,1689656054096 2023-07-18 04:54:15,261 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:15,261 DEBUG [RS:0;jenkins-hbase4:42175] zookeeper.ZKUtil(162): regionserver:42175-0x10176c7bf3c0001, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:15,261 DEBUG [RS:1;jenkins-hbase4:45857] zookeeper.ZKUtil(162): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:15,261 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-18 04:54:15,264 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/table 2023-07-18 04:54:15,265 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-18 04:54:15,266 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:15,268 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740 2023-07-18 04:54:15,269 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740 2023-07-18 04:54:15,274 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-18 04:54:15,276 DEBUG [RS:1;jenkins-hbase4:45857] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 04:54:15,276 DEBUG [RS:0;jenkins-hbase4:42175] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 04:54:15,277 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-18 04:54:15,282 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 04:54:15,283 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64980701, jitterRate=-0.03171209990978241}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-18 04:54:15,283 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-18 04:54:15,283 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-18 04:54:15,283 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-18 04:54:15,283 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-18 04:54:15,283 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-18 04:54:15,283 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-18 04:54:15,284 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-18 04:54:15,284 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-18 04:54:15,287 INFO [RS:1;jenkins-hbase4:45857] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 04:54:15,287 INFO [RS:0;jenkins-hbase4:42175] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 04:54:15,289 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-18 04:54:15,289 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-18 04:54:15,297 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-18 04:54:15,315 INFO [RS:1;jenkins-hbase4:45857] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 04:54:15,315 INFO [RS:0;jenkins-hbase4:42175] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 04:54:15,316 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-18 04:54:15,319 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-18 04:54:15,322 INFO [RS:1;jenkins-hbase4:45857] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 04:54:15,322 INFO [RS:0;jenkins-hbase4:42175] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 04:54:15,322 INFO [RS:1;jenkins-hbase4:45857] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,322 INFO [RS:0;jenkins-hbase4:42175] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,326 INFO [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 04:54:15,326 INFO [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 04:54:15,335 INFO [RS:1;jenkins-hbase4:45857] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,335 INFO [RS:0;jenkins-hbase4:42175] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,335 DEBUG [RS:0;jenkins-hbase4:42175] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,335 DEBUG [RS:1;jenkins-hbase4:45857] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,336 DEBUG [RS:0;jenkins-hbase4:42175] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,336 DEBUG [RS:1;jenkins-hbase4:45857] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,336 DEBUG [RS:0;jenkins-hbase4:42175] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,336 DEBUG [RS:1;jenkins-hbase4:45857] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,336 DEBUG [RS:0;jenkins-hbase4:42175] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,336 DEBUG [RS:1;jenkins-hbase4:45857] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,336 DEBUG [RS:0;jenkins-hbase4:42175] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,336 DEBUG [RS:1;jenkins-hbase4:45857] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,336 DEBUG [RS:0;jenkins-hbase4:42175] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 04:54:15,337 DEBUG [RS:1;jenkins-hbase4:45857] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 04:54:15,337 DEBUG [RS:0;jenkins-hbase4:42175] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,337 DEBUG [RS:1;jenkins-hbase4:45857] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,337 DEBUG [RS:0;jenkins-hbase4:42175] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,337 DEBUG [RS:1;jenkins-hbase4:45857] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,337 DEBUG [RS:0;jenkins-hbase4:42175] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,337 DEBUG [RS:1;jenkins-hbase4:45857] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,337 DEBUG [RS:0;jenkins-hbase4:42175] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,337 DEBUG [RS:1;jenkins-hbase4:45857] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:15,338 INFO [RS:0;jenkins-hbase4:42175] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,339 INFO [RS:1;jenkins-hbase4:45857] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,339 INFO [RS:0;jenkins-hbase4:42175] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,339 INFO [RS:1;jenkins-hbase4:45857] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,339 INFO [RS:0;jenkins-hbase4:42175] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,339 INFO [RS:1;jenkins-hbase4:45857] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,355 INFO [RS:0;jenkins-hbase4:42175] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 04:54:15,355 INFO [RS:1;jenkins-hbase4:45857] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 04:54:15,358 INFO [RS:0;jenkins-hbase4:42175] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42175,1689656054096-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,358 INFO [RS:1;jenkins-hbase4:45857] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45857,1689656054144-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,376 INFO [RS:1;jenkins-hbase4:45857] regionserver.Replication(203): jenkins-hbase4.apache.org,45857,1689656054144 started 2023-07-18 04:54:15,376 INFO [RS:0;jenkins-hbase4:42175] regionserver.Replication(203): jenkins-hbase4.apache.org,42175,1689656054096 started 2023-07-18 04:54:15,376 INFO [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,45857,1689656054144, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45857, sessionid=0x10176c7bf3c0002 2023-07-18 04:54:15,376 INFO [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42175,1689656054096, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42175, sessionid=0x10176c7bf3c0001 2023-07-18 04:54:15,377 DEBUG [RS:0;jenkins-hbase4:42175] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 04:54:15,377 DEBUG [RS:1;jenkins-hbase4:45857] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 04:54:15,377 DEBUG [RS:0;jenkins-hbase4:42175] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42175,1689656054096 2023-07-18 04:54:15,377 DEBUG [RS:1;jenkins-hbase4:45857] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:15,377 DEBUG [RS:0;jenkins-hbase4:42175] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42175,1689656054096' 2023-07-18 04:54:15,378 DEBUG [RS:1;jenkins-hbase4:45857] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45857,1689656054144' 2023-07-18 04:54:15,378 DEBUG [RS:1;jenkins-hbase4:45857] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-18 04:54:15,378 DEBUG [RS:0;jenkins-hbase4:42175] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-18 04:54:15,379 DEBUG [RS:0;jenkins-hbase4:42175] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-18 04:54:15,379 DEBUG [RS:1;jenkins-hbase4:45857] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-18 04:54:15,379 DEBUG [RS:0;jenkins-hbase4:42175] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 04:54:15,379 DEBUG [RS:1;jenkins-hbase4:45857] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 04:54:15,379 DEBUG [RS:0;jenkins-hbase4:42175] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 04:54:15,379 DEBUG [RS:1;jenkins-hbase4:45857] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 04:54:15,380 DEBUG [RS:0;jenkins-hbase4:42175] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42175,1689656054096 2023-07-18 04:54:15,380 DEBUG [RS:1;jenkins-hbase4:45857] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:15,380 DEBUG [RS:1;jenkins-hbase4:45857] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45857,1689656054144' 2023-07-18 04:54:15,380 DEBUG [RS:1;jenkins-hbase4:45857] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-18 04:54:15,380 DEBUG [RS:0;jenkins-hbase4:42175] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42175,1689656054096' 2023-07-18 04:54:15,380 DEBUG [RS:0;jenkins-hbase4:42175] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-18 04:54:15,381 DEBUG [RS:0;jenkins-hbase4:42175] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-18 04:54:15,381 DEBUG [RS:1;jenkins-hbase4:45857] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-18 04:54:15,381 DEBUG [RS:0;jenkins-hbase4:42175] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 04:54:15,381 DEBUG [RS:1;jenkins-hbase4:45857] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 04:54:15,381 INFO [RS:0;jenkins-hbase4:42175] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 04:54:15,381 INFO [RS:1;jenkins-hbase4:45857] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 04:54:15,382 INFO [RS:0;jenkins-hbase4:42175] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 04:54:15,382 INFO [RS:1;jenkins-hbase4:45857] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 04:54:15,472 DEBUG [jenkins-hbase4:44113] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=2, allServersCount=2 2023-07-18 04:54:15,477 DEBUG [jenkins-hbase4:44113] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 04:54:15,485 DEBUG [jenkins-hbase4:44113] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 04:54:15,486 DEBUG [jenkins-hbase4:44113] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 04:54:15,486 DEBUG [jenkins-hbase4:44113] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 04:54:15,489 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42175,1689656054096, state=OPENING 2023-07-18 04:54:15,491 INFO [RS:0;jenkins-hbase4:42175] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:15,491 INFO [RS:1;jenkins-hbase4:45857] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:15,495 INFO [RS:1;jenkins-hbase4:45857] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0, suffix=, logDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144, archiveDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs, maxLogs=10 2023-07-18 04:54:15,495 INFO [RS:0;jenkins-hbase4:42175] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0, suffix=, logDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096, archiveDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs, maxLogs=10 2023-07-18 04:54:15,499 DEBUG [PEWorker-2] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-18 04:54:15,503 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:15,508 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-18 04:54:15,510 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42175,1689656054096}] 2023-07-18 04:54:15,530 INFO [RS:0;jenkins-hbase4:42175] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096/jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0.1689656055499 2023-07-18 04:54:15,530 INFO [RS:1;jenkins-hbase4:45857] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0.1689656055500 2023-07-18 04:54:15,530 DEBUG [RS:0;jenkins-hbase4:42175] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36951,DS-63fc3a07-0ff5-4e01-8767-68d629b125e2,DISK], DatanodeInfoWithStorage[127.0.0.1:34717,DS-15daf992-9329-4b25-9667-a096fb2deda0,DISK]] 2023-07-18 04:54:15,530 DEBUG [RS:1;jenkins-hbase4:45857] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34717,DS-15daf992-9329-4b25-9667-a096fb2deda0,DISK], DatanodeInfoWithStorage[127.0.0.1:36951,DS-63fc3a07-0ff5-4e01-8767-68d629b125e2,DISK]] 2023-07-18 04:54:15,726 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42175,1689656054096 2023-07-18 04:54:15,729 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 04:54:15,733 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38060, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 04:54:15,745 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-18 04:54:15,745 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:15,746 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:15,746 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:15,749 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42175%2C1689656054096.meta, suffix=.meta, logDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096, archiveDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs, maxLogs=10 2023-07-18 04:54:15,767 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096/jenkins-hbase4.apache.org%2C42175%2C1689656054096.meta.1689656055751.meta 2023-07-18 04:54:15,768 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36951,DS-63fc3a07-0ff5-4e01-8767-68d629b125e2,DISK], DatanodeInfoWithStorage[127.0.0.1:34717,DS-15daf992-9329-4b25-9667-a096fb2deda0,DISK]] 2023-07-18 04:54:15,768 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:15,770 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-18 04:54:15,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-18 04:54:15,802 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-18 04:54:15,809 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-18 04:54:15,809 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:15,810 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-18 04:54:15,810 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-18 04:54:15,813 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-18 04:54:15,815 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/info 2023-07-18 04:54:15,815 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/info 2023-07-18 04:54:15,815 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-18 04:54:15,817 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:15,817 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-18 04:54:15,819 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/rep_barrier 2023-07-18 04:54:15,819 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/rep_barrier 2023-07-18 04:54:15,820 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-18 04:54:15,820 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:15,821 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-18 04:54:15,822 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/table 2023-07-18 04:54:15,822 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/table 2023-07-18 04:54:15,823 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-18 04:54:15,823 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:15,825 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740 2023-07-18 04:54:15,828 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740 2023-07-18 04:54:15,832 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-18 04:54:15,835 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-18 04:54:15,836 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74301380, jitterRate=0.10717684030532837}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-18 04:54:15,836 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-18 04:54:15,846 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689656055719 2023-07-18 04:54:15,865 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-18 04:54:15,866 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-18 04:54:15,867 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42175,1689656054096, state=OPEN 2023-07-18 04:54:15,870 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-18 04:54:15,870 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-18 04:54:15,875 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-18 04:54:15,875 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42175,1689656054096 in 360 msec 2023-07-18 04:54:15,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-18 04:54:15,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 579 msec 2023-07-18 04:54:15,887 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 861 msec 2023-07-18 04:54:15,887 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689656055887, completionTime=-1 2023-07-18 04:54:15,887 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=2; waited=0ms, expected min=2 server(s), max=2 server(s), master is running 2023-07-18 04:54:15,888 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-18 04:54:15,949 DEBUG [hconnection-0xf9417ee-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:15,955 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38064, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:15,975 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=2 2023-07-18 04:54:15,975 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689656115975 2023-07-18 04:54:15,975 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689656175975 2023-07-18 04:54:15,975 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 87 msec 2023-07-18 04:54:15,999 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44113,1689656052898-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,999 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44113,1689656052898-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:15,999 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44113,1689656052898-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:16,001 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:44113, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:16,001 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:16,007 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-18 04:54:16,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-18 04:54:16,016 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-18 04:54:16,025 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-18 04:54:16,028 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-18 04:54:16,030 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-18 04:54:16,052 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:16,054 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf empty. 2023-07-18 04:54:16,055 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:16,055 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-18 04:54:16,093 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-18 04:54:16,095 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8750ae11f1669b3efa9c8fcc2b1be9cf, NAME => 'hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp 2023-07-18 04:54:16,116 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:16,116 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 8750ae11f1669b3efa9c8fcc2b1be9cf, disabling compactions & flushes 2023-07-18 04:54:16,116 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:16,117 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:16,117 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. after waiting 0 ms 2023-07-18 04:54:16,117 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:16,117 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:16,117 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 8750ae11f1669b3efa9c8fcc2b1be9cf: 2023-07-18 04:54:16,121 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-18 04:54:16,135 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689656056124"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689656056124"}]},"ts":"1689656056124"} 2023-07-18 04:54:16,161 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-18 04:54:16,163 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-18 04:54:16,168 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689656056163"}]},"ts":"1689656056163"} 2023-07-18 04:54:16,172 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-18 04:54:16,177 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 04:54:16,179 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 04:54:16,179 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 04:54:16,179 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 04:54:16,182 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=8750ae11f1669b3efa9c8fcc2b1be9cf, ASSIGN}] 2023-07-18 04:54:16,184 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=8750ae11f1669b3efa9c8fcc2b1be9cf, ASSIGN 2023-07-18 04:54:16,186 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=8750ae11f1669b3efa9c8fcc2b1be9cf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45857,1689656054144; forceNewPlan=false, retain=false 2023-07-18 04:54:16,338 INFO [jenkins-hbase4:44113] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-18 04:54:16,339 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=8750ae11f1669b3efa9c8fcc2b1be9cf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:16,340 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689656056339"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656056339"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656056339"}]},"ts":"1689656056339"} 2023-07-18 04:54:16,344 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 8750ae11f1669b3efa9c8fcc2b1be9cf, server=jenkins-hbase4.apache.org,45857,1689656054144}] 2023-07-18 04:54:16,499 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:16,499 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 04:54:16,501 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46444, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 04:54:16,508 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:16,509 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:16,512 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1, suffix=, logDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144, archiveDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs, maxLogs=10 2023-07-18 04:54:16,526 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1.1689656056513 2023-07-18 04:54:16,526 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36951,DS-63fc3a07-0ff5-4e01-8767-68d629b125e2,DISK], DatanodeInfoWithStorage[127.0.0.1:34717,DS-15daf992-9329-4b25-9667-a096fb2deda0,DISK]] 2023-07-18 04:54:16,526 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8750ae11f1669b3efa9c8fcc2b1be9cf, NAME => 'hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:16,527 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:16,527 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:16,527 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:16,527 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:16,530 INFO [StoreOpener-8750ae11f1669b3efa9c8fcc2b1be9cf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:16,532 DEBUG [StoreOpener-8750ae11f1669b3efa9c8fcc2b1be9cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf/info 2023-07-18 04:54:16,533 DEBUG [StoreOpener-8750ae11f1669b3efa9c8fcc2b1be9cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf/info 2023-07-18 04:54:16,533 INFO [StoreOpener-8750ae11f1669b3efa9c8fcc2b1be9cf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8750ae11f1669b3efa9c8fcc2b1be9cf columnFamilyName info 2023-07-18 04:54:16,534 INFO [StoreOpener-8750ae11f1669b3efa9c8fcc2b1be9cf-1] regionserver.HStore(310): Store=8750ae11f1669b3efa9c8fcc2b1be9cf/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:16,536 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:16,537 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:16,541 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:16,544 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 04:54:16,545 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8750ae11f1669b3efa9c8fcc2b1be9cf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64104580, jitterRate=-0.04476732015609741}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 04:54:16,546 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8750ae11f1669b3efa9c8fcc2b1be9cf: 2023-07-18 04:54:16,548 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf., pid=6, masterSystemTime=1689656056499 2023-07-18 04:54:16,553 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:16,554 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:16,555 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=8750ae11f1669b3efa9c8fcc2b1be9cf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:16,555 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689656056554"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656056554"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656056554"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656056554"}]},"ts":"1689656056554"} 2023-07-18 04:54:16,563 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-18 04:54:16,564 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 8750ae11f1669b3efa9c8fcc2b1be9cf, server=jenkins-hbase4.apache.org,45857,1689656054144 in 216 msec 2023-07-18 04:54:16,567 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-18 04:54:16,568 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=8750ae11f1669b3efa9c8fcc2b1be9cf, ASSIGN in 382 msec 2023-07-18 04:54:16,569 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-18 04:54:16,570 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689656056570"}]},"ts":"1689656056570"} 2023-07-18 04:54:16,573 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-18 04:54:16,576 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-18 04:54:16,579 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 559 msec 2023-07-18 04:54:16,631 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/namespace 2023-07-18 04:54:16,633 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/namespace 2023-07-18 04:54:16,633 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:16,655 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:16,657 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46446, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:16,674 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-18 04:54:16,695 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-18 04:54:16,701 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 38 msec 2023-07-18 04:54:16,707 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-18 04:54:16,718 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-18 04:54:16,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-07-18 04:54:16,734 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-18 04:54:16,737 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-18 04:54:16,739 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.527sec 2023-07-18 04:54:16,742 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-18 04:54:16,743 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-18 04:54:16,743 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-18 04:54:16,744 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44113,1689656052898-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-18 04:54:16,745 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44113,1689656052898-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-18 04:54:16,753 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-18 04:54:16,780 DEBUG [Listener at localhost/46199] zookeeper.ReadOnlyZKClient(139): Connect 0x06b2f9f8 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:16,787 DEBUG [Listener at localhost/46199] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3ac7f724, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:16,798 DEBUG [hconnection-0x2be9a473-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:16,809 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38070, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:16,818 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,44113,1689656052898 2023-07-18 04:54:16,818 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=4, rsPorts=, rsClass=null, numDataNodes=4, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-18 04:54:16,818 INFO [Listener at localhost/46199] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/cluster_328436eb-65ac-c7b0-f033-ec4bcc6042f1, deleteOnExit=true 2023-07-18 04:54:16,819 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-18 04:54:16,829 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/test.cache.data in system properties and HBase conf 2023-07-18 04:54:16,829 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/hadoop.tmp.dir in system properties and HBase conf 2023-07-18 04:54:16,829 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/hadoop.log.dir in system properties and HBase conf 2023-07-18 04:54:16,830 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-18 04:54:16,830 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-18 04:54:16,830 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-18 04:54:16,830 DEBUG [Listener at localhost/46199] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-18 04:54:16,831 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-18 04:54:16,831 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-18 04:54:16,831 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-18 04:54:16,831 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-18 04:54:16,831 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-18 04:54:16,832 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-18 04:54:16,832 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-18 04:54:16,832 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-18 04:54:16,832 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-18 04:54:16,833 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/nfs.dump.dir in system properties and HBase conf 2023-07-18 04:54:16,833 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/java.io.tmpdir in system properties and HBase conf 2023-07-18 04:54:16,833 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-18 04:54:16,833 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-18 04:54:16,833 INFO [Listener at localhost/46199] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-18 04:54:16,839 WARN [Listener at localhost/46199] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-18 04:54:16,839 WARN [Listener at localhost/46199] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-18 04:54:16,884 WARN [Listener at localhost/46199] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 04:54:16,886 INFO [Listener at localhost/46199] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 04:54:16,891 INFO [Listener at localhost/46199] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/java.io.tmpdir/Jetty_localhost_35749_hdfs____.3sfit8/webapp 2023-07-18 04:54:17,001 INFO [Listener at localhost/46199] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35749 2023-07-18 04:54:17,054 WARN [Listener at localhost/46199] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-18 04:54:17,055 WARN [Listener at localhost/46199] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-18 04:54:17,107 WARN [Listener at localhost/37939] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 04:54:17,128 WARN [Listener at localhost/37939] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-18 04:54:17,131 WARN [Listener at localhost/37939] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 04:54:17,132 INFO [Listener at localhost/37939] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 04:54:17,136 INFO [Listener at localhost/37939] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/java.io.tmpdir/Jetty_localhost_43195_datanode____.fqkl4b/webapp 2023-07-18 04:54:17,229 INFO [Listener at localhost/37939] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43195 2023-07-18 04:54:17,239 WARN [Listener at localhost/45743] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 04:54:17,261 WARN [Listener at localhost/45743] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-18 04:54:17,263 WARN [Listener at localhost/45743] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 04:54:17,265 INFO [Listener at localhost/45743] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 04:54:17,268 INFO [Listener at localhost/45743] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/java.io.tmpdir/Jetty_localhost_36607_datanode____.fb0g4f/webapp 2023-07-18 04:54:17,371 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xef37a9b8d7634fd3: Processing first storage report for DS-cb62bdfa-114c-4e1c-89f3-b40b9c3afb83 from datanode e06ca105-963d-4852-b6fe-f6a9b0a5e830 2023-07-18 04:54:17,371 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xef37a9b8d7634fd3: from storage DS-cb62bdfa-114c-4e1c-89f3-b40b9c3afb83 node DatanodeRegistration(127.0.0.1:40889, datanodeUuid=e06ca105-963d-4852-b6fe-f6a9b0a5e830, infoPort=40747, infoSecurePort=0, ipcPort=45743, storageInfo=lv=-57;cid=testClusterID;nsid=732316756;c=1689656056846), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 04:54:17,371 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xef37a9b8d7634fd3: Processing first storage report for DS-a3746008-3497-4934-a72b-6139ae68c497 from datanode e06ca105-963d-4852-b6fe-f6a9b0a5e830 2023-07-18 04:54:17,371 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xef37a9b8d7634fd3: from storage DS-a3746008-3497-4934-a72b-6139ae68c497 node DatanodeRegistration(127.0.0.1:40889, datanodeUuid=e06ca105-963d-4852-b6fe-f6a9b0a5e830, infoPort=40747, infoSecurePort=0, ipcPort=45743, storageInfo=lv=-57;cid=testClusterID;nsid=732316756;c=1689656056846), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 04:54:17,380 INFO [Listener at localhost/45743] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36607 2023-07-18 04:54:17,390 WARN [Listener at localhost/41783] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 04:54:17,415 WARN [Listener at localhost/41783] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-18 04:54:17,418 WARN [Listener at localhost/41783] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 04:54:17,420 INFO [Listener at localhost/41783] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 04:54:17,424 INFO [Listener at localhost/41783] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/java.io.tmpdir/Jetty_localhost_43813_datanode____mac0ki/webapp 2023-07-18 04:54:17,525 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe51b1decaa2c386f: Processing first storage report for DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347 from datanode 7e19e0ad-89aa-4293-a01b-8c1ee631c073 2023-07-18 04:54:17,525 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe51b1decaa2c386f: from storage DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347 node DatanodeRegistration(127.0.0.1:37603, datanodeUuid=7e19e0ad-89aa-4293-a01b-8c1ee631c073, infoPort=37499, infoSecurePort=0, ipcPort=41783, storageInfo=lv=-57;cid=testClusterID;nsid=732316756;c=1689656056846), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-18 04:54:17,526 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe51b1decaa2c386f: Processing first storage report for DS-6e9d710b-a4e1-4ce0-a9df-4d94890c83c0 from datanode 7e19e0ad-89aa-4293-a01b-8c1ee631c073 2023-07-18 04:54:17,526 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe51b1decaa2c386f: from storage DS-6e9d710b-a4e1-4ce0-a9df-4d94890c83c0 node DatanodeRegistration(127.0.0.1:37603, datanodeUuid=7e19e0ad-89aa-4293-a01b-8c1ee631c073, infoPort=37499, infoSecurePort=0, ipcPort=41783, storageInfo=lv=-57;cid=testClusterID;nsid=732316756;c=1689656056846), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 04:54:17,543 INFO [Listener at localhost/41783] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43813 2023-07-18 04:54:17,555 WARN [Listener at localhost/46409] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 04:54:17,582 WARN [Listener at localhost/46409] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-18 04:54:17,585 WARN [Listener at localhost/46409] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-18 04:54:17,587 INFO [Listener at localhost/46409] log.Slf4jLog(67): jetty-6.1.26 2023-07-18 04:54:17,594 INFO [Listener at localhost/46409] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/java.io.tmpdir/Jetty_localhost_34745_datanode____2rufz2/webapp 2023-07-18 04:54:17,673 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x60dd383a6a0439d6: Processing first storage report for DS-d0c3f250-f9ef-4d3a-8da0-904990569608 from datanode 17506207-b0e0-4651-9028-781a6610874c 2023-07-18 04:54:17,673 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x60dd383a6a0439d6: from storage DS-d0c3f250-f9ef-4d3a-8da0-904990569608 node DatanodeRegistration(127.0.0.1:44317, datanodeUuid=17506207-b0e0-4651-9028-781a6610874c, infoPort=43405, infoSecurePort=0, ipcPort=46409, storageInfo=lv=-57;cid=testClusterID;nsid=732316756;c=1689656056846), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 04:54:17,673 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x60dd383a6a0439d6: Processing first storage report for DS-092050eb-e4df-421e-b70d-076292d5ca04 from datanode 17506207-b0e0-4651-9028-781a6610874c 2023-07-18 04:54:17,674 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x60dd383a6a0439d6: from storage DS-092050eb-e4df-421e-b70d-076292d5ca04 node DatanodeRegistration(127.0.0.1:44317, datanodeUuid=17506207-b0e0-4651-9028-781a6610874c, infoPort=43405, infoSecurePort=0, ipcPort=46409, storageInfo=lv=-57;cid=testClusterID;nsid=732316756;c=1689656056846), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 04:54:17,708 INFO [Listener at localhost/46409] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34745 2023-07-18 04:54:17,717 WARN [Listener at localhost/37011] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-18 04:54:17,821 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfc2b2c082abcf9b4: Processing first storage report for DS-7e9427a5-d2d9-4ece-8da8-f6fb77b8864e from datanode a496e3f3-5645-4243-8cf8-1af079891093 2023-07-18 04:54:17,821 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfc2b2c082abcf9b4: from storage DS-7e9427a5-d2d9-4ece-8da8-f6fb77b8864e node DatanodeRegistration(127.0.0.1:38949, datanodeUuid=a496e3f3-5645-4243-8cf8-1af079891093, infoPort=44981, infoSecurePort=0, ipcPort=37011, storageInfo=lv=-57;cid=testClusterID;nsid=732316756;c=1689656056846), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 04:54:17,821 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfc2b2c082abcf9b4: Processing first storage report for DS-b7a7ed70-cb5e-4dab-88df-36144b09af57 from datanode a496e3f3-5645-4243-8cf8-1af079891093 2023-07-18 04:54:17,821 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfc2b2c082abcf9b4: from storage DS-b7a7ed70-cb5e-4dab-88df-36144b09af57 node DatanodeRegistration(127.0.0.1:38949, datanodeUuid=a496e3f3-5645-4243-8cf8-1af079891093, infoPort=44981, infoSecurePort=0, ipcPort=37011, storageInfo=lv=-57;cid=testClusterID;nsid=732316756;c=1689656056846), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-18 04:54:17,828 DEBUG [Listener at localhost/37011] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219 2023-07-18 04:54:17,828 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:17,829 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:17,868 INFO [Listener at localhost/37011] util.FSUtils(471): Created version file at hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0 with version=8 2023-07-18 04:54:17,869 INFO [Listener at localhost/37011] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/hbase-staging 2023-07-18 04:54:17,870 INFO [Listener at localhost/37011] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 04:54:17,871 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:17,871 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:17,871 INFO [Listener at localhost/37011] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 04:54:17,871 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:17,871 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 04:54:17,871 INFO [Listener at localhost/37011] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 04:54:17,873 INFO [Listener at localhost/37011] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34379 2023-07-18 04:54:17,873 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:17,875 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:17,876 INFO [Listener at localhost/37011] zookeeper.RecoverableZooKeeper(93): Process identifier=master:34379 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:17,879 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:343790x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:17,880 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:34379-0x10176c7bf3c0007 connected 2023-07-18 04:54:17,893 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 04:54:17,894 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:17,894 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 04:54:17,895 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34379 2023-07-18 04:54:17,895 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34379 2023-07-18 04:54:17,895 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34379 2023-07-18 04:54:17,896 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34379 2023-07-18 04:54:17,896 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34379 2023-07-18 04:54:17,896 INFO [Listener at localhost/37011] master.HMaster(444): hbase.rootdir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0, hbase.cluster.distributed=false 2023-07-18 04:54:17,911 INFO [Listener at localhost/37011] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 04:54:17,911 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:17,911 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:17,911 INFO [Listener at localhost/37011] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 04:54:17,911 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:17,911 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 04:54:17,911 INFO [Listener at localhost/37011] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 04:54:17,914 INFO [Listener at localhost/37011] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43893 2023-07-18 04:54:17,915 INFO [Listener at localhost/37011] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 04:54:17,916 DEBUG [Listener at localhost/37011] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 04:54:17,917 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:17,918 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:17,919 INFO [Listener at localhost/37011] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43893 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:17,923 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:438930x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:17,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:43893-0x10176c7bf3c0008 connected 2023-07-18 04:54:17,925 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 04:54:17,925 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:17,926 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 04:54:17,931 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43893 2023-07-18 04:54:17,931 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43893 2023-07-18 04:54:17,934 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43893 2023-07-18 04:54:17,935 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43893 2023-07-18 04:54:17,935 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43893 2023-07-18 04:54:17,947 INFO [Listener at localhost/37011] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 04:54:17,947 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:17,947 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:17,947 INFO [Listener at localhost/37011] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 04:54:17,947 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:17,947 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 04:54:17,947 INFO [Listener at localhost/37011] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 04:54:17,949 INFO [Listener at localhost/37011] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43065 2023-07-18 04:54:17,950 INFO [Listener at localhost/37011] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 04:54:17,951 DEBUG [Listener at localhost/37011] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 04:54:17,952 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:17,953 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:17,954 INFO [Listener at localhost/37011] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43065 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:17,958 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:430650x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:17,959 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:43065-0x10176c7bf3c0009 connected 2023-07-18 04:54:17,960 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 04:54:17,960 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:17,961 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 04:54:17,964 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43065 2023-07-18 04:54:17,964 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43065 2023-07-18 04:54:17,964 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43065 2023-07-18 04:54:17,966 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43065 2023-07-18 04:54:17,966 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43065 2023-07-18 04:54:17,977 INFO [Listener at localhost/37011] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 04:54:17,978 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:17,978 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:17,978 INFO [Listener at localhost/37011] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 04:54:17,978 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:17,978 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 04:54:17,978 INFO [Listener at localhost/37011] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 04:54:17,980 INFO [Listener at localhost/37011] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36167 2023-07-18 04:54:17,980 INFO [Listener at localhost/37011] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 04:54:17,981 DEBUG [Listener at localhost/37011] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 04:54:17,982 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:17,983 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:17,984 INFO [Listener at localhost/37011] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36167 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:17,989 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:361670x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:17,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36167-0x10176c7bf3c000a connected 2023-07-18 04:54:17,991 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 04:54:17,991 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:17,992 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 04:54:17,995 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36167 2023-07-18 04:54:17,995 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36167 2023-07-18 04:54:17,999 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36167 2023-07-18 04:54:17,999 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36167 2023-07-18 04:54:18,000 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36167 2023-07-18 04:54:18,011 INFO [Listener at localhost/37011] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 04:54:18,011 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:18,011 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:18,011 INFO [Listener at localhost/37011] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 04:54:18,011 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:18,011 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 04:54:18,011 INFO [Listener at localhost/37011] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 04:54:18,013 INFO [Listener at localhost/37011] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42145 2023-07-18 04:54:18,014 INFO [Listener at localhost/37011] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 04:54:18,015 DEBUG [Listener at localhost/37011] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 04:54:18,016 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:18,017 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:18,018 INFO [Listener at localhost/37011] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42145 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:18,022 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:421450x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:18,024 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42145-0x10176c7bf3c000b connected 2023-07-18 04:54:18,024 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 04:54:18,025 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:18,025 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 04:54:18,026 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42145 2023-07-18 04:54:18,026 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42145 2023-07-18 04:54:18,030 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42145 2023-07-18 04:54:18,031 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42145 2023-07-18 04:54:18,031 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42145 2023-07-18 04:54:18,033 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,34379,1689656057870 2023-07-18 04:54:18,035 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-18 04:54:18,035 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,34379,1689656057870 2023-07-18 04:54:18,037 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 04:54:18,037 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 04:54:18,037 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 04:54:18,037 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 04:54:18,037 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 04:54:18,039 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:18,040 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-18 04:54:18,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-18 04:54:18,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,34379,1689656057870 from backup master directory 2023-07-18 04:54:18,045 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,34379,1689656057870 2023-07-18 04:54:18,045 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-18 04:54:18,045 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 04:54:18,045 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,34379,1689656057870 2023-07-18 04:54:18,079 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/hbase.id with ID: b492e8cd-f4f9-4591-97c6-7f2edd4114a8 2023-07-18 04:54:18,097 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:18,101 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:18,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x5eb2cab5 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:18,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d151c13, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:18,132 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 04:54:18,133 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-18 04:54:18,133 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:18,134 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:18,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store-tmp 2023-07-18 04:54:18,158 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:18,158 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-18 04:54:18,158 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:18,158 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:18,158 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-18 04:54:18,158 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:18,158 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:18,158 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 04:54:18,160 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,34379,1689656057870 2023-07-18 04:54:18,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:18,165 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34379%2C1689656057870.jenkins-hbase4.apache.org%2C34379%2C1689656057870.regiongroup-0, suffix=, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,34379,1689656057870, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/oldWALs, maxLogs=10 2023-07-18 04:54:18,181 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,34379,1689656057870/jenkins-hbase4.apache.org%2C34379%2C1689656057870.jenkins-hbase4.apache.org%2C34379%2C1689656057870.regiongroup-0.1689656058165 2023-07-18 04:54:18,181 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44317,DS-d0c3f250-f9ef-4d3a-8da0-904990569608,DISK], DatanodeInfoWithStorage[127.0.0.1:37603,DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347,DISK], DatanodeInfoWithStorage[127.0.0.1:40889,DS-cb62bdfa-114c-4e1c-89f3-b40b9c3afb83,DISK]] 2023-07-18 04:54:18,181 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:18,181 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:18,181 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:18,181 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:18,184 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:18,187 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-18 04:54:18,187 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-18 04:54:18,188 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:18,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:18,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:18,195 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:18,198 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 04:54:18,199 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68142306, jitterRate=0.01539948582649231}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 04:54:18,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 04:54:18,199 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-18 04:54:18,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-18 04:54:18,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-18 04:54:18,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-18 04:54:18,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-18 04:54:18,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-07-18 04:54:18,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-18 04:54:18,206 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-18 04:54:18,207 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-18 04:54:18,218 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-18 04:54:18,218 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-18 04:54:18,219 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-18 04:54:18,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-18 04:54:18,219 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-18 04:54:18,222 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:18,223 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-18 04:54:18,224 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-18 04:54:18,225 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-18 04:54:18,226 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 04:54:18,226 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 04:54:18,226 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 04:54:18,226 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 04:54:18,226 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:18,226 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 04:54:18,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,34379,1689656057870, sessionid=0x10176c7bf3c0007, setting cluster-up flag (Was=false) 2023-07-18 04:54:18,233 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:18,242 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-18 04:54:18,243 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34379,1689656057870 2023-07-18 04:54:18,246 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:18,254 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-18 04:54:18,255 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34379,1689656057870 2023-07-18 04:54:18,256 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.hbase-snapshot/.tmp 2023-07-18 04:54:18,259 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-18 04:54:18,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:18,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:18,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:18,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:18,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-18 04:54:18,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 04:54:18,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,263 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689656088263 2023-07-18 04:54:18,263 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-18 04:54:18,264 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-18 04:54:18,264 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-18 04:54:18,264 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-18 04:54:18,264 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-18 04:54:18,264 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-18 04:54:18,264 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,265 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-18 04:54:18,265 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-18 04:54:18,265 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-18 04:54:18,265 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-18 04:54:18,265 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-18 04:54:18,265 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656058265,5,FailOnTimeoutGroup] 2023-07-18 04:54:18,266 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-18 04:54:18,266 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-18 04:54:18,266 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656058265,5,FailOnTimeoutGroup] 2023-07-18 04:54:18,267 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,267 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-18 04:54:18,267 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,267 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,267 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-18 04:54:18,298 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-18 04:54:18,299 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-18 04:54:18,299 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0 2023-07-18 04:54:18,329 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:18,331 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-18 04:54:18,334 INFO [RS:0;jenkins-hbase4:43893] regionserver.HRegionServer(951): ClusterId : b492e8cd-f4f9-4591-97c6-7f2edd4114a8 2023-07-18 04:54:18,334 INFO [RS:2;jenkins-hbase4:36167] regionserver.HRegionServer(951): ClusterId : b492e8cd-f4f9-4591-97c6-7f2edd4114a8 2023-07-18 04:54:18,335 DEBUG [RS:2;jenkins-hbase4:36167] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 04:54:18,335 DEBUG [RS:0;jenkins-hbase4:43893] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 04:54:18,336 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info 2023-07-18 04:54:18,336 INFO [RS:1;jenkins-hbase4:43065] regionserver.HRegionServer(951): ClusterId : b492e8cd-f4f9-4591-97c6-7f2edd4114a8 2023-07-18 04:54:18,336 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-18 04:54:18,337 DEBUG [RS:1;jenkins-hbase4:43065] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 04:54:18,338 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:18,340 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-18 04:54:18,342 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/rep_barrier 2023-07-18 04:54:18,343 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-18 04:54:18,343 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:18,344 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-18 04:54:18,345 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/table 2023-07-18 04:54:18,346 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-18 04:54:18,346 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:18,347 INFO [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(951): ClusterId : b492e8cd-f4f9-4591-97c6-7f2edd4114a8 2023-07-18 04:54:18,349 DEBUG [RS:3;jenkins-hbase4:42145] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 04:54:18,350 DEBUG [RS:0;jenkins-hbase4:43893] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 04:54:18,350 DEBUG [RS:0;jenkins-hbase4:43893] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 04:54:18,351 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740 2023-07-18 04:54:18,351 DEBUG [RS:2;jenkins-hbase4:36167] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 04:54:18,351 DEBUG [RS:2;jenkins-hbase4:36167] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 04:54:18,351 DEBUG [RS:1;jenkins-hbase4:43065] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 04:54:18,352 DEBUG [RS:1;jenkins-hbase4:43065] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 04:54:18,353 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740 2023-07-18 04:54:18,359 DEBUG [RS:3;jenkins-hbase4:42145] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 04:54:18,359 DEBUG [RS:3;jenkins-hbase4:42145] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 04:54:18,359 DEBUG [RS:0;jenkins-hbase4:43893] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 04:54:18,359 DEBUG [RS:2;jenkins-hbase4:36167] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 04:54:18,361 DEBUG [RS:1;jenkins-hbase4:43065] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 04:54:18,362 DEBUG [RS:2;jenkins-hbase4:36167] zookeeper.ReadOnlyZKClient(139): Connect 0x64c721ae to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:18,362 DEBUG [RS:0;jenkins-hbase4:43893] zookeeper.ReadOnlyZKClient(139): Connect 0x09e3115d to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:18,363 DEBUG [RS:1;jenkins-hbase4:43065] zookeeper.ReadOnlyZKClient(139): Connect 0x404d6ac6 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:18,368 DEBUG [RS:3;jenkins-hbase4:42145] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 04:54:18,371 DEBUG [RS:3;jenkins-hbase4:42145] zookeeper.ReadOnlyZKClient(139): Connect 0x00135826 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:18,371 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-18 04:54:18,374 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-18 04:54:18,384 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 04:54:18,385 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73427918, jitterRate=0.09416124224662781}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-18 04:54:18,385 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-18 04:54:18,385 DEBUG [RS:2;jenkins-hbase4:36167] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6a00d346, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:18,385 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-18 04:54:18,385 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-18 04:54:18,385 DEBUG [RS:1;jenkins-hbase4:43065] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@17060fd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:18,385 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-18 04:54:18,385 DEBUG [RS:2;jenkins-hbase4:36167] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6e3f4ade, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 04:54:18,385 DEBUG [RS:0;jenkins-hbase4:43893] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@31b7a2d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:18,385 DEBUG [RS:1;jenkins-hbase4:43065] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1b0b3401, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 04:54:18,385 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-18 04:54:18,386 DEBUG [RS:0;jenkins-hbase4:43893] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4619e42a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 04:54:18,386 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-18 04:54:18,390 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-18 04:54:18,391 DEBUG [RS:3;jenkins-hbase4:42145] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@ed4b654, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:18,391 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-18 04:54:18,391 DEBUG [RS:3;jenkins-hbase4:42145] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@aeb3356, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 04:54:18,393 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-18 04:54:18,393 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-18 04:54:18,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-18 04:54:18,400 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-18 04:54:18,406 DEBUG [RS:2;jenkins-hbase4:36167] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:36167 2023-07-18 04:54:18,407 INFO [RS:2;jenkins-hbase4:36167] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 04:54:18,407 INFO [RS:2;jenkins-hbase4:36167] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 04:54:18,407 DEBUG [RS:2;jenkins-hbase4:36167] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 04:54:18,408 INFO [RS:2;jenkins-hbase4:36167] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,34379,1689656057870 with isa=jenkins-hbase4.apache.org/172.31.14.131:36167, startcode=1689656057977 2023-07-18 04:54:18,408 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-18 04:54:18,409 DEBUG [RS:2;jenkins-hbase4:36167] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 04:54:18,409 DEBUG [RS:0;jenkins-hbase4:43893] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:43893 2023-07-18 04:54:18,409 INFO [RS:0;jenkins-hbase4:43893] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 04:54:18,409 INFO [RS:0;jenkins-hbase4:43893] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 04:54:18,409 DEBUG [RS:0;jenkins-hbase4:43893] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 04:54:18,410 DEBUG [RS:1;jenkins-hbase4:43065] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:43065 2023-07-18 04:54:18,410 INFO [RS:1;jenkins-hbase4:43065] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 04:54:18,410 INFO [RS:0;jenkins-hbase4:43893] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,34379,1689656057870 with isa=jenkins-hbase4.apache.org/172.31.14.131:43893, startcode=1689656057910 2023-07-18 04:54:18,410 INFO [RS:1;jenkins-hbase4:43065] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 04:54:18,411 DEBUG [RS:0;jenkins-hbase4:43893] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 04:54:18,412 DEBUG [RS:1;jenkins-hbase4:43065] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 04:54:18,413 INFO [RS:1;jenkins-hbase4:43065] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,34379,1689656057870 with isa=jenkins-hbase4.apache.org/172.31.14.131:43065, startcode=1689656057946 2023-07-18 04:54:18,413 DEBUG [RS:1;jenkins-hbase4:43065] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 04:54:18,416 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38153, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 04:54:18,416 DEBUG [RS:3;jenkins-hbase4:42145] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase4:42145 2023-07-18 04:54:18,417 INFO [RS:3;jenkins-hbase4:42145] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 04:54:18,417 INFO [RS:3;jenkins-hbase4:42145] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 04:54:18,417 DEBUG [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 04:54:18,419 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34379] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:18,419 INFO [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,34379,1689656057870 with isa=jenkins-hbase4.apache.org/172.31.14.131:42145, startcode=1689656058011 2023-07-18 04:54:18,420 DEBUG [RS:3;jenkins-hbase4:42145] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 04:54:18,420 DEBUG [RS:2;jenkins-hbase4:36167] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0 2023-07-18 04:54:18,420 DEBUG [RS:2;jenkins-hbase4:36167] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37939 2023-07-18 04:54:18,421 DEBUG [RS:2;jenkins-hbase4:36167] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 04:54:18,421 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43655, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 04:54:18,421 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39407, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 04:54:18,421 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34379] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43065,1689656057946 2023-07-18 04:54:18,423 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53355, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 04:54:18,426 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34379] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:18,427 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34379] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:18,427 DEBUG [RS:1;jenkins-hbase4:43065] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0 2023-07-18 04:54:18,427 DEBUG [RS:1;jenkins-hbase4:43065] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37939 2023-07-18 04:54:18,427 DEBUG [RS:1;jenkins-hbase4:43065] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 04:54:18,427 DEBUG [RS:0;jenkins-hbase4:43893] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0 2023-07-18 04:54:18,427 DEBUG [RS:0;jenkins-hbase4:43893] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37939 2023-07-18 04:54:18,428 DEBUG [RS:0;jenkins-hbase4:43893] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 04:54:18,428 DEBUG [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0 2023-07-18 04:54:18,429 DEBUG [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37939 2023-07-18 04:54:18,429 DEBUG [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 04:54:18,429 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:18,431 DEBUG [RS:2;jenkins-hbase4:36167] zookeeper.ZKUtil(162): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:18,431 DEBUG [RS:1;jenkins-hbase4:43065] zookeeper.ZKUtil(162): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43065,1689656057946 2023-07-18 04:54:18,431 WARN [RS:2;jenkins-hbase4:36167] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 04:54:18,432 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43065,1689656057946] 2023-07-18 04:54:18,432 INFO [RS:2;jenkins-hbase4:36167] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:18,432 DEBUG [RS:0;jenkins-hbase4:43893] zookeeper.ZKUtil(162): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:18,431 WARN [RS:1;jenkins-hbase4:43065] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 04:54:18,432 WARN [RS:0;jenkins-hbase4:43893] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 04:54:18,432 INFO [RS:1;jenkins-hbase4:43065] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:18,432 INFO [RS:0;jenkins-hbase4:43893] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:18,432 INFO [RS:2;jenkins-hbase4:36167] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:18,432 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36167,1689656057977] 2023-07-18 04:54:18,432 INFO [RS:0;jenkins-hbase4:43893] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:18,432 INFO [RS:1;jenkins-hbase4:43065] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:18,432 DEBUG [RS:0;jenkins-hbase4:43893] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:18,432 DEBUG [RS:2;jenkins-hbase4:36167] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:18,432 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42145,1689656058011] 2023-07-18 04:54:18,433 DEBUG [RS:3;jenkins-hbase4:42145] zookeeper.ZKUtil(162): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:18,432 DEBUG [RS:1;jenkins-hbase4:43065] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43065,1689656057946 2023-07-18 04:54:18,433 WARN [RS:3;jenkins-hbase4:42145] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 04:54:18,433 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43893,1689656057910] 2023-07-18 04:54:18,433 INFO [RS:3;jenkins-hbase4:42145] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:18,433 INFO [RS:3;jenkins-hbase4:42145] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:18,433 DEBUG [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:18,455 DEBUG [RS:2;jenkins-hbase4:36167] zookeeper.ZKUtil(162): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43065,1689656057946 2023-07-18 04:54:18,456 DEBUG [RS:1;jenkins-hbase4:43065] zookeeper.ZKUtil(162): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43065,1689656057946 2023-07-18 04:54:18,456 DEBUG [RS:2;jenkins-hbase4:36167] zookeeper.ZKUtil(162): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:18,456 DEBUG [RS:0;jenkins-hbase4:43893] zookeeper.ZKUtil(162): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43065,1689656057946 2023-07-18 04:54:18,456 DEBUG [RS:1;jenkins-hbase4:43065] zookeeper.ZKUtil(162): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:18,457 DEBUG [RS:3;jenkins-hbase4:42145] zookeeper.ZKUtil(162): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43065,1689656057946 2023-07-18 04:54:18,457 DEBUG [RS:2;jenkins-hbase4:36167] zookeeper.ZKUtil(162): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:18,457 DEBUG [RS:0;jenkins-hbase4:43893] zookeeper.ZKUtil(162): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:18,457 DEBUG [RS:1;jenkins-hbase4:43065] zookeeper.ZKUtil(162): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:18,457 DEBUG [RS:3;jenkins-hbase4:42145] zookeeper.ZKUtil(162): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:18,457 DEBUG [RS:2;jenkins-hbase4:36167] zookeeper.ZKUtil(162): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:18,458 DEBUG [RS:1;jenkins-hbase4:43065] zookeeper.ZKUtil(162): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:18,458 DEBUG [RS:0;jenkins-hbase4:43893] zookeeper.ZKUtil(162): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:18,458 DEBUG [RS:3;jenkins-hbase4:42145] zookeeper.ZKUtil(162): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:18,459 DEBUG [RS:3;jenkins-hbase4:42145] zookeeper.ZKUtil(162): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:18,459 DEBUG [RS:0;jenkins-hbase4:43893] zookeeper.ZKUtil(162): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:18,459 DEBUG [RS:1;jenkins-hbase4:43065] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 04:54:18,459 INFO [RS:1;jenkins-hbase4:43065] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 04:54:18,459 DEBUG [RS:2;jenkins-hbase4:36167] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 04:54:18,460 DEBUG [RS:3;jenkins-hbase4:42145] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 04:54:18,460 DEBUG [RS:0;jenkins-hbase4:43893] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 04:54:18,462 INFO [RS:3;jenkins-hbase4:42145] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 04:54:18,462 INFO [RS:1;jenkins-hbase4:43065] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 04:54:18,462 INFO [RS:2;jenkins-hbase4:36167] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 04:54:18,462 INFO [RS:0;jenkins-hbase4:43893] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 04:54:18,462 INFO [RS:1;jenkins-hbase4:43065] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 04:54:18,463 INFO [RS:1;jenkins-hbase4:43065] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,463 INFO [RS:1;jenkins-hbase4:43065] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 04:54:18,466 INFO [RS:3;jenkins-hbase4:42145] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 04:54:18,466 INFO [RS:3;jenkins-hbase4:42145] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 04:54:18,466 INFO [RS:3;jenkins-hbase4:42145] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,466 INFO [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 04:54:18,469 INFO [RS:3;jenkins-hbase4:42145] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,469 INFO [RS:0;jenkins-hbase4:43893] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 04:54:18,470 DEBUG [RS:3;jenkins-hbase4:42145] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,470 DEBUG [RS:3;jenkins-hbase4:42145] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,470 INFO [RS:2;jenkins-hbase4:36167] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 04:54:18,470 DEBUG [RS:3;jenkins-hbase4:42145] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,470 DEBUG [RS:3;jenkins-hbase4:42145] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,470 DEBUG [RS:3;jenkins-hbase4:42145] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,470 DEBUG [RS:3;jenkins-hbase4:42145] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 04:54:18,470 DEBUG [RS:3;jenkins-hbase4:42145] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,470 DEBUG [RS:3;jenkins-hbase4:42145] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,470 DEBUG [RS:3;jenkins-hbase4:42145] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,471 DEBUG [RS:3;jenkins-hbase4:42145] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,474 INFO [RS:0;jenkins-hbase4:43893] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 04:54:18,474 INFO [RS:0;jenkins-hbase4:43893] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,475 INFO [RS:2;jenkins-hbase4:36167] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 04:54:18,475 INFO [RS:2;jenkins-hbase4:36167] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,475 INFO [RS:1;jenkins-hbase4:43065] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,476 INFO [RS:0;jenkins-hbase4:43893] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 04:54:18,478 INFO [RS:3;jenkins-hbase4:42145] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,479 DEBUG [RS:1;jenkins-hbase4:43065] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,479 INFO [RS:2;jenkins-hbase4:36167] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 04:54:18,480 DEBUG [RS:1;jenkins-hbase4:43065] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,480 DEBUG [RS:1;jenkins-hbase4:43065] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,479 INFO [RS:3;jenkins-hbase4:42145] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,480 DEBUG [RS:1;jenkins-hbase4:43065] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,480 INFO [RS:3;jenkins-hbase4:42145] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,481 DEBUG [RS:1;jenkins-hbase4:43065] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,481 INFO [RS:0;jenkins-hbase4:43893] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,481 DEBUG [RS:1;jenkins-hbase4:43065] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 04:54:18,481 DEBUG [RS:1;jenkins-hbase4:43065] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,481 DEBUG [RS:1;jenkins-hbase4:43065] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,482 DEBUG [RS:1;jenkins-hbase4:43065] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,482 DEBUG [RS:1;jenkins-hbase4:43065] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,485 INFO [RS:2;jenkins-hbase4:36167] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,486 DEBUG [RS:0;jenkins-hbase4:43893] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,486 DEBUG [RS:0;jenkins-hbase4:43893] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,487 DEBUG [RS:2;jenkins-hbase4:36167] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,487 DEBUG [RS:0;jenkins-hbase4:43893] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,488 DEBUG [RS:2;jenkins-hbase4:36167] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,487 INFO [RS:1;jenkins-hbase4:43065] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,488 DEBUG [RS:2;jenkins-hbase4:36167] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,488 INFO [RS:1;jenkins-hbase4:43065] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,488 DEBUG [RS:2;jenkins-hbase4:36167] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,488 DEBUG [RS:0;jenkins-hbase4:43893] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,488 DEBUG [RS:2;jenkins-hbase4:36167] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,488 DEBUG [RS:0;jenkins-hbase4:43893] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,488 DEBUG [RS:2;jenkins-hbase4:36167] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 04:54:18,488 DEBUG [RS:0;jenkins-hbase4:43893] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 04:54:18,488 DEBUG [RS:2;jenkins-hbase4:36167] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,488 INFO [RS:1;jenkins-hbase4:43065] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,488 DEBUG [RS:2;jenkins-hbase4:36167] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,488 DEBUG [RS:0;jenkins-hbase4:43893] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,489 DEBUG [RS:2;jenkins-hbase4:36167] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,489 DEBUG [RS:0;jenkins-hbase4:43893] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,489 DEBUG [RS:2;jenkins-hbase4:36167] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,489 DEBUG [RS:0;jenkins-hbase4:43893] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,489 DEBUG [RS:0;jenkins-hbase4:43893] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:18,503 INFO [RS:2;jenkins-hbase4:36167] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,503 INFO [RS:2;jenkins-hbase4:36167] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,504 INFO [RS:2;jenkins-hbase4:36167] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,504 INFO [RS:0;jenkins-hbase4:43893] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,504 INFO [RS:0;jenkins-hbase4:43893] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,504 INFO [RS:0;jenkins-hbase4:43893] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,506 INFO [RS:3;jenkins-hbase4:42145] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 04:54:18,506 INFO [RS:3;jenkins-hbase4:42145] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42145,1689656058011-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,510 INFO [RS:1;jenkins-hbase4:43065] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 04:54:18,510 INFO [RS:1;jenkins-hbase4:43065] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43065,1689656057946-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,519 INFO [RS:2;jenkins-hbase4:36167] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 04:54:18,519 INFO [RS:2;jenkins-hbase4:36167] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36167,1689656057977-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,521 INFO [RS:1;jenkins-hbase4:43065] regionserver.Replication(203): jenkins-hbase4.apache.org,43065,1689656057946 started 2023-07-18 04:54:18,521 INFO [RS:1;jenkins-hbase4:43065] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,43065,1689656057946, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43065, sessionid=0x10176c7bf3c0009 2023-07-18 04:54:18,521 DEBUG [RS:1;jenkins-hbase4:43065] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 04:54:18,521 DEBUG [RS:1;jenkins-hbase4:43065] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43065,1689656057946 2023-07-18 04:54:18,521 DEBUG [RS:1;jenkins-hbase4:43065] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43065,1689656057946' 2023-07-18 04:54:18,521 DEBUG [RS:1;jenkins-hbase4:43065] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-18 04:54:18,521 INFO [RS:3;jenkins-hbase4:42145] regionserver.Replication(203): jenkins-hbase4.apache.org,42145,1689656058011 started 2023-07-18 04:54:18,521 INFO [RS:0;jenkins-hbase4:43893] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 04:54:18,521 INFO [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42145,1689656058011, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42145, sessionid=0x10176c7bf3c000b 2023-07-18 04:54:18,521 INFO [RS:0;jenkins-hbase4:43893] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43893,1689656057910-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,522 DEBUG [RS:3;jenkins-hbase4:42145] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 04:54:18,522 DEBUG [RS:3;jenkins-hbase4:42145] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:18,522 DEBUG [RS:3;jenkins-hbase4:42145] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42145,1689656058011' 2023-07-18 04:54:18,522 DEBUG [RS:3;jenkins-hbase4:42145] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-18 04:54:18,522 DEBUG [RS:1;jenkins-hbase4:43065] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-18 04:54:18,523 DEBUG [RS:3;jenkins-hbase4:42145] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-18 04:54:18,524 DEBUG [RS:1;jenkins-hbase4:43065] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 04:54:18,524 DEBUG [RS:1;jenkins-hbase4:43065] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 04:54:18,524 DEBUG [RS:1;jenkins-hbase4:43065] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43065,1689656057946 2023-07-18 04:54:18,524 DEBUG [RS:1;jenkins-hbase4:43065] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43065,1689656057946' 2023-07-18 04:54:18,524 DEBUG [RS:1;jenkins-hbase4:43065] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-18 04:54:18,524 DEBUG [RS:3;jenkins-hbase4:42145] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 04:54:18,524 DEBUG [RS:1;jenkins-hbase4:43065] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-18 04:54:18,524 DEBUG [RS:3;jenkins-hbase4:42145] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 04:54:18,524 DEBUG [RS:3;jenkins-hbase4:42145] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:18,524 DEBUG [RS:3;jenkins-hbase4:42145] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42145,1689656058011' 2023-07-18 04:54:18,524 DEBUG [RS:3;jenkins-hbase4:42145] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-18 04:54:18,525 DEBUG [RS:1;jenkins-hbase4:43065] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 04:54:18,525 INFO [RS:1;jenkins-hbase4:43065] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 04:54:18,525 INFO [RS:1;jenkins-hbase4:43065] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 04:54:18,525 DEBUG [RS:3;jenkins-hbase4:42145] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-18 04:54:18,525 DEBUG [RS:3;jenkins-hbase4:42145] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 04:54:18,525 INFO [RS:3;jenkins-hbase4:42145] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 04:54:18,525 INFO [RS:3;jenkins-hbase4:42145] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 04:54:18,535 INFO [RS:2;jenkins-hbase4:36167] regionserver.Replication(203): jenkins-hbase4.apache.org,36167,1689656057977 started 2023-07-18 04:54:18,535 INFO [RS:2;jenkins-hbase4:36167] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,36167,1689656057977, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36167, sessionid=0x10176c7bf3c000a 2023-07-18 04:54:18,535 DEBUG [RS:2;jenkins-hbase4:36167] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 04:54:18,535 DEBUG [RS:2;jenkins-hbase4:36167] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:18,535 DEBUG [RS:2;jenkins-hbase4:36167] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36167,1689656057977' 2023-07-18 04:54:18,535 DEBUG [RS:2;jenkins-hbase4:36167] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-18 04:54:18,536 DEBUG [RS:2;jenkins-hbase4:36167] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-18 04:54:18,536 DEBUG [RS:2;jenkins-hbase4:36167] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 04:54:18,536 DEBUG [RS:2;jenkins-hbase4:36167] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 04:54:18,536 DEBUG [RS:2;jenkins-hbase4:36167] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:18,536 DEBUG [RS:2;jenkins-hbase4:36167] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36167,1689656057977' 2023-07-18 04:54:18,536 DEBUG [RS:2;jenkins-hbase4:36167] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-18 04:54:18,537 DEBUG [RS:2;jenkins-hbase4:36167] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-18 04:54:18,537 DEBUG [RS:2;jenkins-hbase4:36167] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 04:54:18,537 INFO [RS:2;jenkins-hbase4:36167] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 04:54:18,537 INFO [RS:2;jenkins-hbase4:36167] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 04:54:18,542 INFO [RS:0;jenkins-hbase4:43893] regionserver.Replication(203): jenkins-hbase4.apache.org,43893,1689656057910 started 2023-07-18 04:54:18,542 INFO [RS:0;jenkins-hbase4:43893] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,43893,1689656057910, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43893, sessionid=0x10176c7bf3c0008 2023-07-18 04:54:18,542 DEBUG [RS:0;jenkins-hbase4:43893] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 04:54:18,542 DEBUG [RS:0;jenkins-hbase4:43893] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:18,542 DEBUG [RS:0;jenkins-hbase4:43893] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43893,1689656057910' 2023-07-18 04:54:18,542 DEBUG [RS:0;jenkins-hbase4:43893] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-18 04:54:18,543 DEBUG [RS:0;jenkins-hbase4:43893] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-18 04:54:18,543 DEBUG [RS:0;jenkins-hbase4:43893] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 04:54:18,543 DEBUG [RS:0;jenkins-hbase4:43893] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 04:54:18,543 DEBUG [RS:0;jenkins-hbase4:43893] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:18,543 DEBUG [RS:0;jenkins-hbase4:43893] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43893,1689656057910' 2023-07-18 04:54:18,543 DEBUG [RS:0;jenkins-hbase4:43893] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-18 04:54:18,544 DEBUG [RS:0;jenkins-hbase4:43893] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-18 04:54:18,544 DEBUG [RS:0;jenkins-hbase4:43893] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 04:54:18,544 INFO [RS:0;jenkins-hbase4:43893] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 04:54:18,544 INFO [RS:0;jenkins-hbase4:43893] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 04:54:18,559 DEBUG [jenkins-hbase4:34379] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=4, allServersCount=4 2023-07-18 04:54:18,559 DEBUG [jenkins-hbase4:34379] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 04:54:18,562 DEBUG [jenkins-hbase4:34379] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 04:54:18,562 DEBUG [jenkins-hbase4:34379] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 04:54:18,562 DEBUG [jenkins-hbase4:34379] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-18 04:54:18,562 DEBUG [jenkins-hbase4:34379] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-18 04:54:18,562 DEBUG [jenkins-hbase4:34379] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 04:54:18,563 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42145,1689656058011, state=OPENING 2023-07-18 04:54:18,567 DEBUG [PEWorker-2] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-18 04:54:18,568 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:18,569 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-18 04:54:18,569 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42145,1689656058011}] 2023-07-18 04:54:18,625 INFO [RS:1;jenkins-hbase4:43065] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:18,625 INFO [RS:3;jenkins-hbase4:42145] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:18,628 INFO [RS:3;jenkins-hbase4:42145] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42145%2C1689656058011.jenkins-hbase4.apache.org%2C42145%2C1689656058011.regiongroup-0, suffix=, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs, maxLogs=10 2023-07-18 04:54:18,628 INFO [RS:1;jenkins-hbase4:43065] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43065%2C1689656057946.jenkins-hbase4.apache.org%2C43065%2C1689656057946.regiongroup-0, suffix=, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43065,1689656057946, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs, maxLogs=10 2023-07-18 04:54:18,639 INFO [RS:2;jenkins-hbase4:36167] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:18,644 INFO [RS:0;jenkins-hbase4:43893] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:18,646 INFO [RS:2;jenkins-hbase4:36167] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36167%2C1689656057977.jenkins-hbase4.apache.org%2C36167%2C1689656057977.regiongroup-0, suffix=, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,36167,1689656057977, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs, maxLogs=10 2023-07-18 04:54:18,653 INFO [RS:0;jenkins-hbase4:43893] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43893%2C1689656057910.jenkins-hbase4.apache.org%2C43893%2C1689656057910.regiongroup-0, suffix=, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43893,1689656057910, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs, maxLogs=10 2023-07-18 04:54:18,675 INFO [RS:1;jenkins-hbase4:43065] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43065,1689656057946/jenkins-hbase4.apache.org%2C43065%2C1689656057946.jenkins-hbase4.apache.org%2C43065%2C1689656057946.regiongroup-0.1689656058635 2023-07-18 04:54:18,685 DEBUG [RS:1;jenkins-hbase4:43065] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40889,DS-cb62bdfa-114c-4e1c-89f3-b40b9c3afb83,DISK], DatanodeInfoWithStorage[127.0.0.1:38949,DS-7e9427a5-d2d9-4ece-8da8-f6fb77b8864e,DISK], DatanodeInfoWithStorage[127.0.0.1:37603,DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347,DISK]] 2023-07-18 04:54:18,699 INFO [RS:3;jenkins-hbase4:42145] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011/jenkins-hbase4.apache.org%2C42145%2C1689656058011.jenkins-hbase4.apache.org%2C42145%2C1689656058011.regiongroup-0.1689656058636 2023-07-18 04:54:18,700 DEBUG [RS:3;jenkins-hbase4:42145] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37603,DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347,DISK], DatanodeInfoWithStorage[127.0.0.1:44317,DS-d0c3f250-f9ef-4d3a-8da0-904990569608,DISK], DatanodeInfoWithStorage[127.0.0.1:40889,DS-cb62bdfa-114c-4e1c-89f3-b40b9c3afb83,DISK]] 2023-07-18 04:54:18,726 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:18,727 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 04:54:18,736 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44204, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 04:54:18,743 INFO [RS:2;jenkins-hbase4:36167] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,36167,1689656057977/jenkins-hbase4.apache.org%2C36167%2C1689656057977.jenkins-hbase4.apache.org%2C36167%2C1689656057977.regiongroup-0.1689656058650 2023-07-18 04:54:18,743 DEBUG [RS:2;jenkins-hbase4:36167] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38949,DS-7e9427a5-d2d9-4ece-8da8-f6fb77b8864e,DISK], DatanodeInfoWithStorage[127.0.0.1:37603,DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347,DISK], DatanodeInfoWithStorage[127.0.0.1:40889,DS-cb62bdfa-114c-4e1c-89f3-b40b9c3afb83,DISK]] 2023-07-18 04:54:18,747 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-18 04:54:18,747 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:18,747 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:18,748 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:18,748 INFO [RS:0;jenkins-hbase4:43893] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43893,1689656057910/jenkins-hbase4.apache.org%2C43893%2C1689656057910.jenkins-hbase4.apache.org%2C43893%2C1689656057910.regiongroup-0.1689656058667 2023-07-18 04:54:18,748 DEBUG [RS:0;jenkins-hbase4:43893] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37603,DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347,DISK], DatanodeInfoWithStorage[127.0.0.1:38949,DS-7e9427a5-d2d9-4ece-8da8-f6fb77b8864e,DISK], DatanodeInfoWithStorage[127.0.0.1:40889,DS-cb62bdfa-114c-4e1c-89f3-b40b9c3afb83,DISK]] 2023-07-18 04:54:18,750 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42145%2C1689656058011.meta, suffix=.meta, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs, maxLogs=10 2023-07-18 04:54:18,770 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011/jenkins-hbase4.apache.org%2C42145%2C1689656058011.meta.1689656058751.meta 2023-07-18 04:54:18,770 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37603,DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347,DISK], DatanodeInfoWithStorage[127.0.0.1:40889,DS-cb62bdfa-114c-4e1c-89f3-b40b9c3afb83,DISK], DatanodeInfoWithStorage[127.0.0.1:44317,DS-d0c3f250-f9ef-4d3a-8da0-904990569608,DISK]] 2023-07-18 04:54:18,771 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:18,771 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-18 04:54:18,771 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-18 04:54:18,772 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-18 04:54:18,772 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-18 04:54:18,772 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:18,772 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-18 04:54:18,772 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-18 04:54:18,774 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-18 04:54:18,776 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info 2023-07-18 04:54:18,776 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info 2023-07-18 04:54:18,776 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-18 04:54:18,777 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:18,777 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-18 04:54:18,779 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/rep_barrier 2023-07-18 04:54:18,779 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/rep_barrier 2023-07-18 04:54:18,779 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-18 04:54:18,780 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:18,780 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-18 04:54:18,781 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/table 2023-07-18 04:54:18,781 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/table 2023-07-18 04:54:18,781 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-18 04:54:18,782 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:18,785 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740 2023-07-18 04:54:18,787 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740 2023-07-18 04:54:18,791 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-18 04:54:18,793 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-18 04:54:18,794 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67019680, jitterRate=-0.0013289451599121094}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-18 04:54:18,794 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-18 04:54:18,797 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689656058726 2023-07-18 04:54:18,804 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-18 04:54:18,805 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-18 04:54:18,806 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42145,1689656058011, state=OPEN 2023-07-18 04:54:18,808 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-18 04:54:18,808 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-18 04:54:18,813 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-18 04:54:18,813 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42145,1689656058011 in 239 msec 2023-07-18 04:54:18,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-18 04:54:18,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 420 msec 2023-07-18 04:54:18,821 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 562 msec 2023-07-18 04:54:18,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689656058821, completionTime=-1 2023-07-18 04:54:18,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=4; waited=0ms, expected min=4 server(s), max=4 server(s), master is running 2023-07-18 04:54:18,821 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-18 04:54:18,825 DEBUG [hconnection-0x6672967c-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:18,828 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44216, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:18,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=4 2023-07-18 04:54:18,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689656118830 2023-07-18 04:54:18,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689656178830 2023-07-18 04:54:18,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 9 msec 2023-07-18 04:54:18,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34379,1689656057870-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34379,1689656057870-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34379,1689656057870-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:34379, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:18,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-18 04:54:18,840 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-18 04:54:18,842 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-18 04:54:18,843 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-18 04:54:18,844 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-18 04:54:18,846 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-18 04:54:18,848 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:18,849 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e empty. 2023-07-18 04:54:18,850 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:18,850 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-18 04:54:18,873 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-18 04:54:18,875 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 383bb30520997d1bb825dc5e5f31999e, NAME => 'hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp 2023-07-18 04:54:18,891 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:18,891 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 383bb30520997d1bb825dc5e5f31999e, disabling compactions & flushes 2023-07-18 04:54:18,891 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:18,891 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:18,891 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. after waiting 0 ms 2023-07-18 04:54:18,891 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:18,891 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:18,892 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 383bb30520997d1bb825dc5e5f31999e: 2023-07-18 04:54:18,895 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-18 04:54:18,897 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689656058897"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689656058897"}]},"ts":"1689656058897"} 2023-07-18 04:54:18,901 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-18 04:54:18,903 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-18 04:54:18,903 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689656058903"}]},"ts":"1689656058903"} 2023-07-18 04:54:18,906 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-18 04:54:18,910 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 04:54:18,911 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 04:54:18,911 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 04:54:18,911 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-18 04:54:18,911 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-18 04:54:18,911 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 04:54:18,911 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=383bb30520997d1bb825dc5e5f31999e, ASSIGN}] 2023-07-18 04:54:18,914 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=383bb30520997d1bb825dc5e5f31999e, ASSIGN 2023-07-18 04:54:18,916 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=383bb30520997d1bb825dc5e5f31999e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42145,1689656058011; forceNewPlan=false, retain=false 2023-07-18 04:54:19,066 INFO [jenkins-hbase4:34379] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-18 04:54:19,068 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=383bb30520997d1bb825dc5e5f31999e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:19,068 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689656059068"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656059068"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656059068"}]},"ts":"1689656059068"} 2023-07-18 04:54:19,072 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 383bb30520997d1bb825dc5e5f31999e, server=jenkins-hbase4.apache.org,42145,1689656058011}] 2023-07-18 04:54:19,233 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:19,233 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:19,235 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42145%2C1689656058011.jenkins-hbase4.apache.org%2C42145%2C1689656058011.regiongroup-1, suffix=, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs, maxLogs=10 2023-07-18 04:54:19,252 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011/jenkins-hbase4.apache.org%2C42145%2C1689656058011.jenkins-hbase4.apache.org%2C42145%2C1689656058011.regiongroup-1.1689656059236 2023-07-18 04:54:19,252 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40889,DS-cb62bdfa-114c-4e1c-89f3-b40b9c3afb83,DISK], DatanodeInfoWithStorage[127.0.0.1:44317,DS-d0c3f250-f9ef-4d3a-8da0-904990569608,DISK], DatanodeInfoWithStorage[127.0.0.1:37603,DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347,DISK]] 2023-07-18 04:54:19,252 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 383bb30520997d1bb825dc5e5f31999e, NAME => 'hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:19,252 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:19,252 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:19,252 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:19,252 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:19,254 INFO [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:19,258 DEBUG [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/info 2023-07-18 04:54:19,258 DEBUG [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/info 2023-07-18 04:54:19,260 INFO [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 383bb30520997d1bb825dc5e5f31999e columnFamilyName info 2023-07-18 04:54:19,261 INFO [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] regionserver.HStore(310): Store=383bb30520997d1bb825dc5e5f31999e/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:19,262 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:19,264 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:19,268 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:19,271 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 04:54:19,272 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 383bb30520997d1bb825dc5e5f31999e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71352413, jitterRate=0.06323380768299103}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 04:54:19,272 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 383bb30520997d1bb825dc5e5f31999e: 2023-07-18 04:54:19,275 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e., pid=6, masterSystemTime=1689656059226 2023-07-18 04:54:19,279 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:19,279 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:19,280 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=383bb30520997d1bb825dc5e5f31999e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:19,280 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689656059280"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656059280"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656059280"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656059280"}]},"ts":"1689656059280"} 2023-07-18 04:54:19,287 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-18 04:54:19,287 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 383bb30520997d1bb825dc5e5f31999e, server=jenkins-hbase4.apache.org,42145,1689656058011 in 212 msec 2023-07-18 04:54:19,291 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-18 04:54:19,292 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=383bb30520997d1bb825dc5e5f31999e, ASSIGN in 376 msec 2023-07-18 04:54:19,293 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-18 04:54:19,293 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689656059293"}]},"ts":"1689656059293"} 2023-07-18 04:54:19,295 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-18 04:54:19,298 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-18 04:54:19,301 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 458 msec 2023-07-18 04:54:19,343 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/namespace 2023-07-18 04:54:19,348 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/namespace 2023-07-18 04:54:19,348 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:19,355 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-18 04:54:19,369 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-18 04:54:19,375 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 20 msec 2023-07-18 04:54:19,388 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-18 04:54:19,400 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-18 04:54:19,407 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 19 msec 2023-07-18 04:54:19,412 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-18 04:54:19,417 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-18 04:54:19,417 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.372sec 2023-07-18 04:54:19,417 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-18 04:54:19,417 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-18 04:54:19,417 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-18 04:54:19,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34379,1689656057870-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-18 04:54:19,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34379,1689656057870-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-18 04:54:19,422 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-18 04:54:19,434 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(139): Connect 0x7cf2ef6a to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:19,441 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@70846748, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:19,443 DEBUG [hconnection-0x3b436e8-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:19,446 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44226, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:19,448 INFO [Listener at localhost/37011] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,34379,1689656057870 2023-07-18 04:54:19,461 DEBUG [Listener at localhost/37011] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-18 04:54:19,464 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55166, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-18 04:54:19,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 04:54:19,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-18 04:54:19,483 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-18 04:54:19,485 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-18 04:54:19,488 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:19,488 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-18 04:54:19,489 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a empty. 2023-07-18 04:54:19,490 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:19,490 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-18 04:54:19,502 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 04:54:19,518 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-18 04:54:19,520 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => db11400df50c3fb2fe7bab3791162e0a, NAME => 't1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp 2023-07-18 04:54:19,534 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:19,534 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing db11400df50c3fb2fe7bab3791162e0a, disabling compactions & flushes 2023-07-18 04:54:19,534 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:19,534 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:19,534 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. after waiting 0 ms 2023-07-18 04:54:19,534 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:19,534 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:19,535 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for db11400df50c3fb2fe7bab3791162e0a: 2023-07-18 04:54:19,539 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-18 04:54:19,541 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656059541"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689656059541"}]},"ts":"1689656059541"} 2023-07-18 04:54:19,544 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-18 04:54:19,545 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-18 04:54:19,545 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689656059545"}]},"ts":"1689656059545"} 2023-07-18 04:54:19,547 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-18 04:54:19,551 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 04:54:19,552 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 04:54:19,553 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 04:54:19,553 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 04:54:19,553 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=db11400df50c3fb2fe7bab3791162e0a, ASSIGN}] 2023-07-18 04:54:19,555 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=db11400df50c3fb2fe7bab3791162e0a, ASSIGN 2023-07-18 04:54:19,556 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=db11400df50c3fb2fe7bab3791162e0a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45857,1689656054144; forceNewPlan=false, retain=false 2023-07-18 04:54:19,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 04:54:19,707 INFO [jenkins-hbase4:44113] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-18 04:54:19,708 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=db11400df50c3fb2fe7bab3791162e0a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:19,709 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656059708"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656059708"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656059708"}]},"ts":"1689656059708"} 2023-07-18 04:54:19,712 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure db11400df50c3fb2fe7bab3791162e0a, server=jenkins-hbase4.apache.org,45857,1689656054144}] 2023-07-18 04:54:19,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 04:54:19,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:19,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => db11400df50c3fb2fe7bab3791162e0a, NAME => 't1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:19,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:19,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:19,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:19,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:19,876 INFO [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:19,878 DEBUG [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/cf1 2023-07-18 04:54:19,878 DEBUG [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/cf1 2023-07-18 04:54:19,878 INFO [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db11400df50c3fb2fe7bab3791162e0a columnFamilyName cf1 2023-07-18 04:54:19,879 INFO [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] regionserver.HStore(310): Store=db11400df50c3fb2fe7bab3791162e0a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:19,879 INFO [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:19,881 DEBUG [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/norep 2023-07-18 04:54:19,881 DEBUG [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/norep 2023-07-18 04:54:19,882 INFO [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db11400df50c3fb2fe7bab3791162e0a columnFamilyName norep 2023-07-18 04:54:19,882 INFO [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] regionserver.HStore(310): Store=db11400df50c3fb2fe7bab3791162e0a/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:19,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:19,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:19,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 04:54:19,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:19,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 04:54:19,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened db11400df50c3fb2fe7bab3791162e0a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61385041, jitterRate=-0.08529160916805267}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 04:54:19,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for db11400df50c3fb2fe7bab3791162e0a: 2023-07-18 04:54:19,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a., pid=11, masterSystemTime=1689656059866 2023-07-18 04:54:19,900 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:19,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:19,901 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=db11400df50c3fb2fe7bab3791162e0a, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:19,901 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656059901"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656059901"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656059901"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656059901"}]},"ts":"1689656059901"} 2023-07-18 04:54:19,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-18 04:54:19,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure db11400df50c3fb2fe7bab3791162e0a, server=jenkins-hbase4.apache.org,45857,1689656054144 in 192 msec 2023-07-18 04:54:19,910 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-18 04:54:19,912 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=db11400df50c3fb2fe7bab3791162e0a, ASSIGN in 354 msec 2023-07-18 04:54:19,912 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-18 04:54:19,913 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689656059913"}]},"ts":"1689656059913"} 2023-07-18 04:54:19,915 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-18 04:54:19,918 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-18 04:54:19,921 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 440 msec 2023-07-18 04:54:20,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 04:54:20,114 INFO [Listener at localhost/37011] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-18 04:54:20,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 04:54:20,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-18 04:54:20,121 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-18 04:54:20,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-18 04:54:20,124 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-18 04:54:20,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 04:54:20,126 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:20,127 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60 empty. 2023-07-18 04:54:20,127 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:20,127 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-18 04:54:20,146 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-18 04:54:20,147 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 7f6a4e55b8779eead03ae75b46b98c60, NAME => 't2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.tmp 2023-07-18 04:54:20,160 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:20,160 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 7f6a4e55b8779eead03ae75b46b98c60, disabling compactions & flushes 2023-07-18 04:54:20,160 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:20,160 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:20,160 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. after waiting 0 ms 2023-07-18 04:54:20,160 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:20,160 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:20,160 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 7f6a4e55b8779eead03ae75b46b98c60: 2023-07-18 04:54:20,166 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-18 04:54:20,168 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656060168"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689656060168"}]},"ts":"1689656060168"} 2023-07-18 04:54:20,171 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-18 04:54:20,173 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-18 04:54:20,173 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689656060173"}]},"ts":"1689656060173"} 2023-07-18 04:54:20,175 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-18 04:54:20,179 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 04:54:20,181 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 04:54:20,181 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 04:54:20,181 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 04:54:20,181 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=7f6a4e55b8779eead03ae75b46b98c60, ASSIGN}] 2023-07-18 04:54:20,184 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=7f6a4e55b8779eead03ae75b46b98c60, ASSIGN 2023-07-18 04:54:20,185 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=7f6a4e55b8779eead03ae75b46b98c60, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45857,1689656054144; forceNewPlan=false, retain=false 2023-07-18 04:54:20,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 04:54:20,335 INFO [jenkins-hbase4:44113] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-18 04:54:20,337 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=7f6a4e55b8779eead03ae75b46b98c60, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:20,337 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656060337"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656060337"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656060337"}]},"ts":"1689656060337"} 2023-07-18 04:54:20,340 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 7f6a4e55b8779eead03ae75b46b98c60, server=jenkins-hbase4.apache.org,45857,1689656054144}] 2023-07-18 04:54:20,376 INFO [jenkins-hbase4:42175Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 04:54:20,376 INFO [jenkins-hbase4:45857Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 04:54:20,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 04:54:20,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:20,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7f6a4e55b8779eead03ae75b46b98c60, NAME => 't2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:20,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:20,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:20,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:20,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:20,500 INFO [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:20,502 DEBUG [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/cf1 2023-07-18 04:54:20,502 DEBUG [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/cf1 2023-07-18 04:54:20,503 INFO [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7f6a4e55b8779eead03ae75b46b98c60 columnFamilyName cf1 2023-07-18 04:54:20,503 INFO [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] regionserver.HStore(310): Store=7f6a4e55b8779eead03ae75b46b98c60/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:20,504 INFO [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:20,505 DEBUG [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/norep 2023-07-18 04:54:20,506 DEBUG [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/norep 2023-07-18 04:54:20,506 INFO [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7f6a4e55b8779eead03ae75b46b98c60 columnFamilyName norep 2023-07-18 04:54:20,507 INFO [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] regionserver.HStore(310): Store=7f6a4e55b8779eead03ae75b46b98c60/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:20,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:20,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:20,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 04:54:20,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:20,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 04:54:20,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7f6a4e55b8779eead03ae75b46b98c60; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66784465, jitterRate=-0.004833921790122986}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 04:54:20,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7f6a4e55b8779eead03ae75b46b98c60: 2023-07-18 04:54:20,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60., pid=14, masterSystemTime=1689656060493 2023-07-18 04:54:20,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:20,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:20,526 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=7f6a4e55b8779eead03ae75b46b98c60, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:20,526 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656060525"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656060525"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656060525"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656060525"}]},"ts":"1689656060525"} 2023-07-18 04:54:20,532 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-18 04:54:20,532 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 7f6a4e55b8779eead03ae75b46b98c60, server=jenkins-hbase4.apache.org,45857,1689656054144 in 189 msec 2023-07-18 04:54:20,535 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-18 04:54:20,536 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=7f6a4e55b8779eead03ae75b46b98c60, ASSIGN in 351 msec 2023-07-18 04:54:20,537 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-18 04:54:20,538 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689656060537"}]},"ts":"1689656060537"} 2023-07-18 04:54:20,540 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-18 04:54:20,543 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-18 04:54:20,546 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 427 msec 2023-07-18 04:54:20,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 04:54:20,730 INFO [Listener at localhost/37011] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-18 04:54:20,733 DEBUG [Listener at localhost/37011] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-18 04:54:20,738 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37538, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-18 04:54:20,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34379] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 04:54:20,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34379] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-18 04:54:20,747 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-18 04:54:20,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34379] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-18 04:54:20,749 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-18 04:54:20,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34379] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 04:54:20,751 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:20,751 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0 empty. 2023-07-18 04:54:20,752 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:20,752 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-18 04:54:20,770 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-18 04:54:20,772 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => a2e71ceab1ce1a583f43e863ff7cd0a0, NAME => 't1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp 2023-07-18 04:54:20,785 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:20,785 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing a2e71ceab1ce1a583f43e863ff7cd0a0, disabling compactions & flushes 2023-07-18 04:54:20,785 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:20,785 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:20,785 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. after waiting 0 ms 2023-07-18 04:54:20,785 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:20,785 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:20,785 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for a2e71ceab1ce1a583f43e863ff7cd0a0: 2023-07-18 04:54:20,790 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-18 04:54:20,794 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656060794"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689656060794"}]},"ts":"1689656060794"} 2023-07-18 04:54:20,796 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-18 04:54:20,798 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-18 04:54:20,798 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689656060798"}]},"ts":"1689656060798"} 2023-07-18 04:54:20,800 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-18 04:54:20,805 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 04:54:20,806 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 04:54:20,806 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 04:54:20,806 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-18 04:54:20,806 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-18 04:54:20,806 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 04:54:20,807 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=a2e71ceab1ce1a583f43e863ff7cd0a0, ASSIGN}] 2023-07-18 04:54:20,809 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=a2e71ceab1ce1a583f43e863ff7cd0a0, ASSIGN 2023-07-18 04:54:20,810 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=a2e71ceab1ce1a583f43e863ff7cd0a0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42145,1689656058011; forceNewPlan=false, retain=false 2023-07-18 04:54:20,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34379] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 04:54:20,961 INFO [jenkins-hbase4:34379] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-18 04:54:20,963 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=a2e71ceab1ce1a583f43e863ff7cd0a0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:20,963 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656060963"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656060963"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656060963"}]},"ts":"1689656060963"} 2023-07-18 04:54:20,966 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure a2e71ceab1ce1a583f43e863ff7cd0a0, server=jenkins-hbase4.apache.org,42145,1689656058011}] 2023-07-18 04:54:21,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34379] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 04:54:21,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:21,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a2e71ceab1ce1a583f43e863ff7cd0a0, NAME => 't1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:21,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:21,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:21,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:21,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:21,131 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:21,132 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1 2023-07-18 04:54:21,133 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1 2023-07-18 04:54:21,133 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2e71ceab1ce1a583f43e863ff7cd0a0 columnFamilyName cf1 2023-07-18 04:54:21,134 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(310): Store=a2e71ceab1ce1a583f43e863ff7cd0a0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:21,134 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:21,135 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/norep 2023-07-18 04:54:21,135 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/norep 2023-07-18 04:54:21,136 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2e71ceab1ce1a583f43e863ff7cd0a0 columnFamilyName norep 2023-07-18 04:54:21,136 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(310): Store=a2e71ceab1ce1a583f43e863ff7cd0a0/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:21,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:21,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:21,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 04:54:21,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:21,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 04:54:21,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a2e71ceab1ce1a583f43e863ff7cd0a0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66752817, jitterRate=-0.0053055137395858765}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 04:54:21,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a2e71ceab1ce1a583f43e863ff7cd0a0: 2023-07-18 04:54:21,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0., pid=11, masterSystemTime=1689656061119 2023-07-18 04:54:21,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:21,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:21,154 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=a2e71ceab1ce1a583f43e863ff7cd0a0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:21,154 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656061154"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656061154"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656061154"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656061154"}]},"ts":"1689656061154"} 2023-07-18 04:54:21,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-18 04:54:21,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure a2e71ceab1ce1a583f43e863ff7cd0a0, server=jenkins-hbase4.apache.org,42145,1689656058011 in 191 msec 2023-07-18 04:54:21,164 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-18 04:54:21,164 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=a2e71ceab1ce1a583f43e863ff7cd0a0, ASSIGN in 354 msec 2023-07-18 04:54:21,165 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-18 04:54:21,165 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689656061165"}]},"ts":"1689656061165"} 2023-07-18 04:54:21,168 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-18 04:54:21,171 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-18 04:54:21,174 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 431 msec 2023-07-18 04:54:21,338 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 04:54:21,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34379] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-18 04:54:21,356 INFO [Listener at localhost/37011] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-18 04:54:21,358 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34379] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 04:54:21,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34379] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-18 04:54:21,365 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-18 04:54:21,365 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34379] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-18 04:54:21,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34379] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 04:54:21,366 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-18 04:54:21,380 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:21,381 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577 empty. 2023-07-18 04:54:21,381 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:21,381 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-18 04:54:21,435 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-18 04:54:21,437 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1b9326fa16ae8497bb1df12a88dfd577, NAME => 't2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.tmp 2023-07-18 04:54:21,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34379] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 04:54:21,476 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:21,476 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 1b9326fa16ae8497bb1df12a88dfd577, disabling compactions & flushes 2023-07-18 04:54:21,476 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:21,476 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:21,476 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. after waiting 0 ms 2023-07-18 04:54:21,476 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:21,476 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:21,476 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 1b9326fa16ae8497bb1df12a88dfd577: 2023-07-18 04:54:21,482 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-18 04:54:21,483 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656061483"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689656061483"}]},"ts":"1689656061483"} 2023-07-18 04:54:21,485 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-18 04:54:21,489 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-18 04:54:21,489 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689656061489"}]},"ts":"1689656061489"} 2023-07-18 04:54:21,492 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-18 04:54:21,497 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-18 04:54:21,502 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-18 04:54:21,502 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-18 04:54:21,502 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-18 04:54:21,502 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-18 04:54:21,502 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-18 04:54:21,508 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=1b9326fa16ae8497bb1df12a88dfd577, ASSIGN}] 2023-07-18 04:54:21,512 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=1b9326fa16ae8497bb1df12a88dfd577, ASSIGN 2023-07-18 04:54:21,513 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=1b9326fa16ae8497bb1df12a88dfd577, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42145,1689656058011; forceNewPlan=false, retain=false 2023-07-18 04:54:21,577 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-18 04:54:21,578 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-18 04:54:21,578 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-18 04:54:21,579 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-18 04:54:21,664 INFO [jenkins-hbase4:34379] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-18 04:54:21,666 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=1b9326fa16ae8497bb1df12a88dfd577, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:21,666 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656061666"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656061666"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656061666"}]},"ts":"1689656061666"} 2023-07-18 04:54:21,669 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 1b9326fa16ae8497bb1df12a88dfd577, server=jenkins-hbase4.apache.org,42145,1689656058011}] 2023-07-18 04:54:21,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34379] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 04:54:21,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:21,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1b9326fa16ae8497bb1df12a88dfd577, NAME => 't2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:21,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:21,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:21,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:21,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:21,833 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:21,839 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1 2023-07-18 04:54:21,839 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1 2023-07-18 04:54:21,839 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b9326fa16ae8497bb1df12a88dfd577 columnFamilyName cf1 2023-07-18 04:54:21,840 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(310): Store=1b9326fa16ae8497bb1df12a88dfd577/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:21,840 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:21,842 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/norep 2023-07-18 04:54:21,842 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/norep 2023-07-18 04:54:21,842 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b9326fa16ae8497bb1df12a88dfd577 columnFamilyName norep 2023-07-18 04:54:21,843 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(310): Store=1b9326fa16ae8497bb1df12a88dfd577/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:21,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:21,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:21,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 04:54:21,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:21,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-18 04:54:21,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1b9326fa16ae8497bb1df12a88dfd577; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74831444, jitterRate=0.11507540941238403}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 04:54:21,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1b9326fa16ae8497bb1df12a88dfd577: 2023-07-18 04:54:21,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577., pid=14, masterSystemTime=1689656061822 2023-07-18 04:54:21,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:21,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:21,856 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=1b9326fa16ae8497bb1df12a88dfd577, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:21,856 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656061856"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656061856"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656061856"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656061856"}]},"ts":"1689656061856"} 2023-07-18 04:54:21,863 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-18 04:54:21,863 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 1b9326fa16ae8497bb1df12a88dfd577, server=jenkins-hbase4.apache.org,42145,1689656058011 in 190 msec 2023-07-18 04:54:21,866 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-18 04:54:21,866 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=1b9326fa16ae8497bb1df12a88dfd577, ASSIGN in 355 msec 2023-07-18 04:54:21,867 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-18 04:54:21,868 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689656061868"}]},"ts":"1689656061868"} 2023-07-18 04:54:21,870 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-18 04:54:21,873 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-18 04:54:21,875 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 515 msec 2023-07-18 04:54:21,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34379] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-18 04:54:21,972 INFO [Listener at localhost/37011] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-18 04:54:21,973 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(139): Connect 0x1142dfc9 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:21,981 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@355e9c12, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:21,981 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(139): Connect 0x3c29d558 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:21,997 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3b92e983, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:22,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.HMaster(3464): Client=jenkins//172.31.14.131 creating replication peer, id=1, config=clusterKey=127.0.0.1:63236:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false, state=ENABLED 2023-07-18 04:54:22,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:PRE_PEER_MODIFICATION; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-18 04:54:22,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-18 04:54:22,043 INFO [PEWorker-1] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-18 04:54:22,044 INFO [PEWorker-1] zookeeper.RecoverableZooKeeper(93): Process identifier=org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@131b5217check-peer-cluster-id connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:22,050 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@131b5217check-peer-cluster-id0x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:22,051 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@131b5217check-peer-cluster-id-0x10176c7bf3c0014 connected 2023-07-18 04:54:22,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-18 04:54:22,154 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@131b5217check-peer-cluster-id-0x10176c7bf3c0014, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:22,162 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:22,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=17, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-07-18 04:54:22,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-18 04:54:22,343 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42175] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=16 2023-07-18 04:54:22,343 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45857] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=17 2023-07-18 04:54:22,345 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-18 04:54:22,345 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-18 04:54:22,364 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-18 04:54:22,364 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-18 04:54:22,392 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-18 04:54:22,392 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-18 04:54:22,419 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=17 2023-07-18 04:54:22,419 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=16 2023-07-18 04:54:22,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] master.HMaster(3641): Remote procedure done, pid=16 2023-07-18 04:54:22,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44113] master.HMaster(3641): Remote procedure done, pid=17 2023-07-18 04:54:22,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44113] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,42175,1689656054096 suceeded 2023-07-18 04:54:22,423 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44113] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,45857,1689656054144 suceeded 2023-07-18 04:54:22,430 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 251 msec 2023-07-18 04:54:22,431 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-07-18 04:54:22,431 INFO [PEWorker-2] replication.AddPeerProcedure(102): Successfully added ENABLED peer 1, config clusterKey=127.0.0.1:63236:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false 2023-07-18 04:54:22,431 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 252 msec 2023-07-18 04:54:22,434 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure in 415 msec 2023-07-18 04:54:22,449 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x6f23048d to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:22,458 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@4b84463e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:22,460 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:22,467 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x6c5e2a11 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:22,468 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:22,470 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10176c7bf3c0016 connected 2023-07-18 04:54:22,472 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=4b2ea01c-df7d-4e18-8185-4db3e50f17db to cluster=b492e8cd-f4f9-4591-97c6-7f2edd4114a8 2023-07-18 04:54:22,475 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@434b8c5e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:22,477 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:22,479 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1 2023-07-18 04:54:22,486 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:22,487 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10176c7bf3c0018 connected 2023-07-18 04:54:22,488 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=1) is replicating from cluster=4b2ea01c-df7d-4e18-8185-4db3e50f17db to cluster=b492e8cd-f4f9-4591-97c6-7f2edd4114a8 2023-07-18 04:54:22,488 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0 2023-07-18 04:54:22,488 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 04:54:22,488 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 04:54:22,490 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0 2023-07-18 04:54:22,490 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1 2023-07-18 04:54:22,495 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 04:54:22,495 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0 2023-07-18 04:54:22,499 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0 2023-07-18 04:54:22,579 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:22,583 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-18 04:54:22,583 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 04:54:22,583 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-18 04:54:22,583 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0.1689656055500, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-18 04:54:22,583 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-18 04:54:22,583 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 04:54:22,583 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-18 04:54:22,583 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1.1689656056513, lastWalPosition=901, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-18 04:54:22,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44113] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-18 04:54:22,637 DEBUG [Listener at localhost/37011] replication.TestReplicationSyncUpTool(91): putAndReplicateRows 2023-07-18 04:54:22,639 DEBUG [hconnection-0x6a09e0cc-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:22,645 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42182, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:22,650 DEBUG [Listener at localhost/37011] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:22,653 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46098, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:22,690 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:22,697 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-18 04:54:22,701 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 1834 2023-07-18 04:54:22,701 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 14 WAL entries eligible for replication 2023-07-18 04:54:22,701 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 1834 2023-07-18 04:54:22,702 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/db11400df50c3fb2fe7bab3791162e0a/4=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/5=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/6=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/7=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/8=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/9=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/10=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/11=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/12=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/13=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/14=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/15=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/16=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/17=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0.1689656055500, lastWalPosition=1834, nbRowKeys=14, nbHFiles=0, heapSize=2912, lastSeqIds={}, endOfFile=false,usedBufferSize=2912] 2023-07-18 04:54:22,718 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 04:54:22,728 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59138, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-18 04:54:22,732 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=36167] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 04:54:22,732 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=36167] zookeeper.ReadOnlyZKClient(139): Connect 0x35bc84dc to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:22,738 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=36167] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@7016fe18, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:22,748 DEBUG [hconnection-0x645dfae5-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:22,749 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60732, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-18 04:54:22,799 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:22,807 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 901 2023-07-18 04:54:22,818 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=36167] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 04:54:22,829 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 7819 2023-07-18 04:54:22,829 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 57 WAL entries eligible for replication 2023-07-18 04:54:22,829 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 7819 2023-07-18 04:54:22,830 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/db11400df50c3fb2fe7bab3791162e0a/18=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/19=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/20=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/21=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/22=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/23=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/24=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/25=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/26=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/27=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/28=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/29=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/30=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/31=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/32=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/33=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/34=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/35=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/36=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/37=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/38=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/39=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/40=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/41=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/42=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/43=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/44=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/45=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/46=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/47=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/48=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/49=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/50=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/51=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/52=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/53=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/54=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/55=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/56=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/57=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/58=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/59=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/60=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/61=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/62=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/63=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/64=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/65=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/66=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/67=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/68=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/69=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/70=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/71=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/72=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/73=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/74=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0.1689656055500, lastWalPosition=7819, nbRowKeys=57, nbHFiles=0, heapSize=11856, lastSeqIds={}, endOfFile=false,usedBufferSize=11856] 2023-07-18 04:54:22,836 DEBUG [SinkThread-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 04:54:22,842 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49474, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-18 04:54:22,846 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43893] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 04:54:22,847 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43893] zookeeper.ReadOnlyZKClient(139): Connect 0x55ea7a70 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:22,857 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43893] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@4b0f5ef1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:22,861 DEBUG [hconnection-0x64752af3-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:22,862 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60740, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-18 04:54:22,876 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43893] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 04:54:22,914 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:22,926 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 2246 2023-07-18 04:54:22,926 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 13 WAL entries eligible for replication 2023-07-18 04:54:22,926 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 2246 2023-07-18 04:54:22,926 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/4=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/5=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/6=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/7=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/8=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/9=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/10=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/11=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/12=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/13=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/14=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/15=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/16=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1.1689656056513, lastWalPosition=2246, nbRowKeys=13, nbHFiles=0, heapSize=2704, lastSeqIds={}, endOfFile=false,usedBufferSize=2704] 2023-07-18 04:54:22,932 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=36167] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 04:54:22,943 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=36167] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 04:54:22,946 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:22,946 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 29 WAL entries eligible for replication 2023-07-18 04:54:22,947 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:22,947 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/db11400df50c3fb2fe7bab3791162e0a/75=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/76=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/77=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/78=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/79=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/80=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/81=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/82=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/83=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/84=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/85=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/86=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/87=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/88=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/89=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/90=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/91=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/92=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/93=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/94=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/95=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/96=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/97=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/98=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/99=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/100=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/101=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/102=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/103=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0.1689656055500, lastWalPosition=10975, nbRowKeys=29, nbHFiles=0, heapSize=6032, lastSeqIds={}, endOfFile=false,usedBufferSize=6032] 2023-07-18 04:54:22,950 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=36167] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 04:54:22,955 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=36167] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 04:54:23,024 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:23,045 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 9911 2023-07-18 04:54:23,046 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 73 WAL entries eligible for replication 2023-07-18 04:54:23,046 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 9911 2023-07-18 04:54:23,047 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/17=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/18=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/19=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/20=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/21=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/22=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/23=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/24=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/25=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/26=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/27=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/28=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/29=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/30=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/31=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/32=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/33=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/34=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/35=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/36=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/37=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/38=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/39=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/40=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/41=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/42=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/43=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/44=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/45=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/46=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/47=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/48=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/49=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/50=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/51=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/52=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/53=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/54=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/55=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/56=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/57=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/58=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/59=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/60=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/61=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/62=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/63=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/64=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/65=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/66=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/67=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/68=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/69=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/70=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/71=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/72=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/73=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/74=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/75=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/76=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/77=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/78=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/79=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/80=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/81=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/82=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/83=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/84=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/85=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/86=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/87=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/88=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/89=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1.1689656056513, lastWalPosition=9911, nbRowKeys=73, nbHFiles=0, heapSize=15184, lastSeqIds={}, endOfFile=false,usedBufferSize=15184] 2023-07-18 04:54:23,052 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=36167] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 04:54:23,057 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:23,063 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=36167] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 04:54:23,080 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-18 04:54:23,081 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-18 04:54:23,134 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:23,169 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:23,170 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 18269 2023-07-18 04:54:23,170 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 78 WAL entries eligible for replication 2023-07-18 04:54:23,171 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 18269 2023-07-18 04:54:23,171 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/90=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/91=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/92=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/93=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/94=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/95=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/96=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/97=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/98=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/99=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/100=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/101=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/102=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/103=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/104=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/105=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/106=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/107=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/108=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/109=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/110=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/111=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/112=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/113=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/114=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/115=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/116=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/117=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/118=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/119=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/120=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/121=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/122=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/123=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/124=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/125=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/126=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/127=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/128=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/129=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/130=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/131=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/132=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/133=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/134=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/135=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/136=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/137=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/138=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/139=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/140=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/141=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/142=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/143=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/144=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/145=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/146=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/147=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/148=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/149=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/150=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/151=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/152=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/153=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/154=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/155=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/156=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/157=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/158=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/159=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/160=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/161=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/162=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/163=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/164=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/165=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/166=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/167=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1.1689656056513, lastWalPosition=18269, nbRowKeys=78, nbHFiles=0, heapSize=16224, lastSeqIds={}, endOfFile=false,usedBufferSize=16224] 2023-07-18 04:54:23,173 DEBUG [SinkThread-5] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 04:54:23,178 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56160, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=AdminService 2023-07-18 04:54:23,181 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43065] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 04:54:23,182 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43065] zookeeper.ReadOnlyZKClient(139): Connect 0x51a72024 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:23,190 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43065] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@5f60c460, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:23,193 DEBUG [hconnection-0x5b5f4b54-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:23,194 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60756, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-07-18 04:54:23,209 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43065] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 04:54:23,244 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:23,286 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:23,292 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:23,292 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 36 WAL entries eligible for replication 2023-07-18 04:54:23,292 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:23,292 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/168=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/169=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/170=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/171=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/172=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/173=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/174=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/175=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/176=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/177=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/178=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/179=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/180=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/181=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/182=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/183=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/184=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/185=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/186=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/187=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/188=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/189=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/190=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/191=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/192=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/193=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/194=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/195=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/196=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/197=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/198=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/199=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/200=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/201=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/202=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/203=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1.1689656056513, lastWalPosition=22269, nbRowKeys=36, nbHFiles=0, heapSize=7488, lastSeqIds={}, endOfFile=false,usedBufferSize=7488] 2023-07-18 04:54:23,295 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43065] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 04:54:23,300 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43065] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 04:54:23,353 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:23,396 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:23,401 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:23,464 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:23,506 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:23,512 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:23,521 INFO [jenkins-hbase4:43065Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 04:54:23,521 INFO [jenkins-hbase4:43065Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 114 2023-07-18 04:54:23,521 INFO [jenkins-hbase4:42145Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 04:54:23,535 INFO [jenkins-hbase4:36167Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 04:54:23,535 INFO [jenkins-hbase4:36167Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 129 2023-07-18 04:54:23,539 INFO [jenkins-hbase4:43893Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 04:54:23,540 INFO [jenkins-hbase4:43893Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 57 2023-07-18 04:54:23,572 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:23,615 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:23,625 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:23,685 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:23,722 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:23,732 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:23,738 DEBUG [hconnection-0x7aed4cef-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:23,743 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60758, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:23,772 DEBUG [Listener at localhost/37011] replication.TestReplicationSyncUpTool(145): mimicSyncUpAfterDelete 2023-07-18 04:54:23,773 INFO [Listener at localhost/37011] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-18 04:54:23,773 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7cf2ef6a to 127.0.0.1:63236 2023-07-18 04:54:23,773 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:23,774 DEBUG [Listener at localhost/37011] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-18 04:54:23,774 DEBUG [Listener at localhost/37011] util.JVMClusterUtil(257): Found active master hash=1547175025, stopped=false 2023-07-18 04:54:23,774 INFO [Listener at localhost/37011] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,34379,1689656057870 2023-07-18 04:54:23,778 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 04:54:23,778 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 04:54:23,779 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 04:54:23,779 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 04:54:23,779 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 04:54:23,779 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:23,779 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:23,779 INFO [Listener at localhost/37011] procedure2.ProcedureExecutor(629): Stopping 2023-07-18 04:54:23,780 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5eb2cab5 to 127.0.0.1:63236 2023-07-18 04:54:23,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:23,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:23,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:23,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:23,780 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:23,782 INFO [Listener at localhost/37011] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43893,1689656057910' ***** 2023-07-18 04:54:23,782 INFO [Listener at localhost/37011] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 04:54:23,782 INFO [Listener at localhost/37011] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43065,1689656057946' ***** 2023-07-18 04:54:23,783 INFO [Listener at localhost/37011] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 04:54:23,783 INFO [Listener at localhost/37011] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36167,1689656057977' ***** 2023-07-18 04:54:23,783 INFO [Listener at localhost/37011] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 04:54:23,783 INFO [Listener at localhost/37011] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42145,1689656058011' ***** 2023-07-18 04:54:23,783 INFO [RS:1;jenkins-hbase4:43065] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 04:54:23,783 INFO [Listener at localhost/37011] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 04:54:23,785 INFO [RS:1;jenkins-hbase4:43065] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 04:54:23,783 INFO [RS:2;jenkins-hbase4:36167] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 04:54:23,785 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 04:54:23,785 INFO [RS:2;jenkins-hbase4:36167] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 04:54:23,783 INFO [RS:0;jenkins-hbase4:43893] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 04:54:23,786 INFO [RS:2;jenkins-hbase4:36167] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 04:54:23,786 INFO [RS:0;jenkins-hbase4:43893] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 04:54:23,786 INFO [RS:2;jenkins-hbase4:36167] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:23,787 DEBUG [RS:2;jenkins-hbase4:36167] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x64c721ae to 127.0.0.1:63236 2023-07-18 04:54:23,787 DEBUG [RS:2;jenkins-hbase4:36167] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:23,787 INFO [RS:2;jenkins-hbase4:36167] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36167,1689656057977; all regions closed. 2023-07-18 04:54:23,786 INFO [RS:3;jenkins-hbase4:42145] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 04:54:23,787 INFO [RS:3;jenkins-hbase4:42145] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 04:54:23,787 INFO [RS:3;jenkins-hbase4:42145] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 04:54:23,787 INFO [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(3305): Received CLOSE for a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:23,788 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 04:54:23,785 INFO [RS:1;jenkins-hbase4:43065] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 04:54:23,786 INFO [RS:0;jenkins-hbase4:43893] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 04:54:23,788 INFO [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(3305): Received CLOSE for 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:23,786 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 04:54:23,788 INFO [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(3305): Received CLOSE for 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:23,786 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 04:54:23,789 INFO [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:23,788 INFO [RS:0;jenkins-hbase4:43893] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:23,788 INFO [RS:1;jenkins-hbase4:43065] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43065,1689656057946 2023-07-18 04:54:23,789 DEBUG [RS:1;jenkins-hbase4:43065] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x404d6ac6 to 127.0.0.1:63236 2023-07-18 04:54:23,789 DEBUG [RS:0;jenkins-hbase4:43893] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x09e3115d to 127.0.0.1:63236 2023-07-18 04:54:23,790 DEBUG [RS:0;jenkins-hbase4:43893] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:23,790 INFO [RS:0;jenkins-hbase4:43893] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43893,1689656057910; all regions closed. 2023-07-18 04:54:23,789 DEBUG [RS:3;jenkins-hbase4:42145] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x00135826 to 127.0.0.1:63236 2023-07-18 04:54:23,792 DEBUG [RS:3;jenkins-hbase4:42145] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:23,792 INFO [RS:3;jenkins-hbase4:42145] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 04:54:23,792 INFO [RS:3;jenkins-hbase4:42145] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 04:54:23,792 INFO [RS:3;jenkins-hbase4:42145] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 04:54:23,792 INFO [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-18 04:54:23,791 DEBUG [RS:1;jenkins-hbase4:43065] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:23,792 INFO [RS:1;jenkins-hbase4:43065] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43065,1689656057946; all regions closed. 2023-07-18 04:54:23,790 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:23,796 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:23,797 INFO [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-18 04:54:23,798 DEBUG [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(1478): Online Regions={a2e71ceab1ce1a583f43e863ff7cd0a0=t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0., 1588230740=hbase:meta,,1.1588230740, 1b9326fa16ae8497bb1df12a88dfd577=t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577., 383bb30520997d1bb825dc5e5f31999e=hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.} 2023-07-18 04:54:23,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a2e71ceab1ce1a583f43e863ff7cd0a0, disabling compactions & flushes 2023-07-18 04:54:23,801 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43065,1689656057946 2023-07-18 04:54:23,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:23,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:23,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. after waiting 0 ms 2023-07-18 04:54:23,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:23,802 DEBUG [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(1504): Waiting on 1588230740, 1b9326fa16ae8497bb1df12a88dfd577, 383bb30520997d1bb825dc5e5f31999e, a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:23,810 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing a2e71ceab1ce1a583f43e863ff7cd0a0 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-18 04:54:23,799 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:23,799 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-18 04:54:23,804 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:23,812 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:23,812 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-18 04:54:23,813 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-18 04:54:23,813 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-18 04:54:23,813 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-18 04:54:23,815 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:23,815 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.65 KB heapSize=7.06 KB 2023-07-18 04:54:23,825 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:23,837 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:23,840 DEBUG [RS:0;jenkins-hbase4:43893] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs 2023-07-18 04:54:23,840 INFO [RS:0;jenkins-hbase4:43893] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C43893%2C1689656057910.jenkins-hbase4.apache.org%2C43893%2C1689656057910.regiongroup-0:(num 1689656058667) 2023-07-18 04:54:23,840 DEBUG [RS:0;jenkins-hbase4:43893] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:23,840 INFO [RS:0;jenkins-hbase4:43893] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:23,841 INFO [RS:0;jenkins-hbase4:43893] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-18 04:54:23,894 INFO [RS:0;jenkins-hbase4:43893] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 04:54:23,898 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 04:54:23,897 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:23,897 DEBUG [RS:2;jenkins-hbase4:36167] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs 2023-07-18 04:54:23,898 INFO [RS:0;jenkins-hbase4:43893] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 04:54:23,899 INFO [RS:0;jenkins-hbase4:43893] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 04:54:23,899 INFO [RS:2;jenkins-hbase4:36167] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C36167%2C1689656057977.jenkins-hbase4.apache.org%2C36167%2C1689656057977.regiongroup-0:(num 1689656058650) 2023-07-18 04:54:23,899 DEBUG [RS:2;jenkins-hbase4:36167] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:23,899 INFO [RS:2;jenkins-hbase4:36167] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:23,899 INFO [RS:2;jenkins-hbase4:36167] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-18 04:54:23,900 INFO [RS:2;jenkins-hbase4:36167] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 04:54:23,900 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 04:54:23,901 DEBUG [RS:0;jenkins-hbase4:43893] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x55ea7a70 to 127.0.0.1:63236 2023-07-18 04:54:23,901 DEBUG [RS:0;jenkins-hbase4:43893] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:23,900 INFO [RS:2;jenkins-hbase4:36167] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 04:54:23,902 INFO [RS:0;jenkins-hbase4:43893] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43893 2023-07-18 04:54:23,902 INFO [RS:2;jenkins-hbase4:36167] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 04:54:23,906 DEBUG [RS:2;jenkins-hbase4:36167] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x35bc84dc to 127.0.0.1:63236 2023-07-18 04:54:23,907 DEBUG [RS:2;jenkins-hbase4:36167] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:23,908 INFO [RS:2;jenkins-hbase4:36167] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36167 2023-07-18 04:54:23,929 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:23,929 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:23,929 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:23,930 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10176c7bf3c0016, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:23,930 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:23,929 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:23,930 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:23,930 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 04:54:23,929 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:23,930 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,36167,1689656057977 2023-07-18 04:54:23,930 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:23,930 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:23,930 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:23,935 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:23,930 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43893,1689656057910 2023-07-18 04:54:23,933 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:23,935 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36167,1689656057977] 2023-07-18 04:54:23,935 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36167,1689656057977; numProcessing=1 2023-07-18 04:54:23,936 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,36167,1689656057977 already deleted, retry=false 2023-07-18 04:54:23,936 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36167,1689656057977 expired; onlineServers=3 2023-07-18 04:54:23,936 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43893,1689656057910] 2023-07-18 04:54:23,936 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43893,1689656057910; numProcessing=2 2023-07-18 04:54:23,938 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,43893,1689656057910 already deleted, retry=false 2023-07-18 04:54:23,938 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43893,1689656057910 expired; onlineServers=2 2023-07-18 04:54:23,945 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:23,982 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/.tmp/info/d057a108d301417792b3fe3bc3fbdfc0 2023-07-18 04:54:23,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/.tmp/cf1/cc4f63e41e55476a976035dc43508488 2023-07-18 04:54:24,009 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:24,010 DEBUG [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(1504): Waiting on 1588230740, 1b9326fa16ae8497bb1df12a88dfd577, 383bb30520997d1bb825dc5e5f31999e, a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:24,027 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/.tmp/cf1/cc4f63e41e55476a976035dc43508488 as hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1/cc4f63e41e55476a976035dc43508488 2023-07-18 04:54:24,049 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1/cc4f63e41e55476a976035dc43508488, entries=100, sequenceid=7, filesize=8.3 K 2023-07-18 04:54:24,050 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:24,051 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for a2e71ceab1ce1a583f43e863ff7cd0a0 in 249ms, sequenceid=7, compaction requested=false 2023-07-18 04:54:24,058 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:24,062 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/.tmp/table/bd05a84742aa4c3d88ab2415d20b934f 2023-07-18 04:54:24,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-07-18 04:54:24,065 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:24,065 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a2e71ceab1ce1a583f43e863ff7cd0a0: 2023-07-18 04:54:24,065 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:24,066 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1b9326fa16ae8497bb1df12a88dfd577, disabling compactions & flushes 2023-07-18 04:54:24,066 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:24,066 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:24,066 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. after waiting 0 ms 2023-07-18 04:54:24,066 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:24,066 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1b9326fa16ae8497bb1df12a88dfd577 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-18 04:54:24,073 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/.tmp/info/d057a108d301417792b3fe3bc3fbdfc0 as hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info/d057a108d301417792b3fe3bc3fbdfc0 2023-07-18 04:54:24,080 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info/d057a108d301417792b3fe3bc3fbdfc0, entries=30, sequenceid=19, filesize=8.1 K 2023-07-18 04:54:24,084 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/.tmp/table/bd05a84742aa4c3d88ab2415d20b934f as hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/table/bd05a84742aa4c3d88ab2415d20b934f 2023-07-18 04:54:24,087 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=8 (bloomFilter=true), to=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/.tmp/cf1/bbbbe1a481884a64ac20a4d86fe979a7 2023-07-18 04:54:24,092 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/table/bd05a84742aa4c3d88ab2415d20b934f, entries=6, sequenceid=19, filesize=4.8 K 2023-07-18 04:54:24,093 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.65 KB/3738, heapSize ~6.78 KB/6944, currentSize=0 B/0 for 1588230740 in 278ms, sequenceid=19, compaction requested=false 2023-07-18 04:54:24,102 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/.tmp/cf1/bbbbe1a481884a64ac20a4d86fe979a7 as hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1/bbbbe1a481884a64ac20a4d86fe979a7 2023-07-18 04:54:24,105 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-18 04:54:24,106 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-18 04:54:24,106 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-18 04:54:24,106 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-18 04:54:24,107 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-18 04:54:24,108 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1/bbbbe1a481884a64ac20a4d86fe979a7, entries=200, sequenceid=8, filesize=12.2 K 2023-07-18 04:54:24,109 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 1b9326fa16ae8497bb1df12a88dfd577 in 43ms, sequenceid=8, compaction requested=false 2023-07-18 04:54:24,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/recovered.edits/11.seqid, newMaxSeqId=11, maxSeqId=1 2023-07-18 04:54:24,118 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:24,118 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:24,118 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1b9326fa16ae8497bb1df12a88dfd577: 2023-07-18 04:54:24,118 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:24,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 383bb30520997d1bb825dc5e5f31999e, disabling compactions & flushes 2023-07-18 04:54:24,119 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:24,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:24,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. after waiting 0 ms 2023-07-18 04:54:24,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:24,119 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 383bb30520997d1bb825dc5e5f31999e 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-18 04:54:24,133 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/.tmp/info/fad48ef599d54b4b956ee92103e35a9a 2023-07-18 04:54:24,140 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/.tmp/info/fad48ef599d54b4b956ee92103e35a9a as hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/info/fad48ef599d54b4b956ee92103e35a9a 2023-07-18 04:54:24,146 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/info/fad48ef599d54b4b956ee92103e35a9a, entries=2, sequenceid=6, filesize=4.8 K 2023-07-18 04:54:24,147 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 383bb30520997d1bb825dc5e5f31999e in 28ms, sequenceid=6, compaction requested=false 2023-07-18 04:54:24,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-18 04:54:24,157 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:24,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 383bb30520997d1bb825dc5e5f31999e: 2023-07-18 04:54:24,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:24,157 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:24,163 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:24,210 INFO [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42145,1689656058011; all regions closed. 2023-07-18 04:54:24,210 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:24,225 DEBUG [RS:3;jenkins-hbase4:42145] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs 2023-07-18 04:54:24,225 INFO [RS:3;jenkins-hbase4:42145] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C42145%2C1689656058011.meta:.meta(num 1689656058751) 2023-07-18 04:54:24,226 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:24,227 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:24,233 DEBUG [RS:3;jenkins-hbase4:42145] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs 2023-07-18 04:54:24,233 INFO [RS:3;jenkins-hbase4:42145] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C42145%2C1689656058011.jenkins-hbase4.apache.org%2C42145%2C1689656058011.regiongroup-0:(num 1689656058636) 2023-07-18 04:54:24,234 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:24,234 DEBUG [RS:1;jenkins-hbase4:43065] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs 2023-07-18 04:54:24,234 INFO [RS:1;jenkins-hbase4:43065] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C43065%2C1689656057946.jenkins-hbase4.apache.org%2C43065%2C1689656057946.regiongroup-0:(num 1689656058635) 2023-07-18 04:54:24,234 DEBUG [RS:1;jenkins-hbase4:43065] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:24,234 INFO [RS:1;jenkins-hbase4:43065] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:24,235 INFO [RS:1;jenkins-hbase4:43065] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-18 04:54:24,235 INFO [RS:1;jenkins-hbase4:43065] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 04:54:24,235 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 04:54:24,235 INFO [RS:1;jenkins-hbase4:43065] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 04:54:24,235 INFO [RS:1;jenkins-hbase4:43065] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 04:54:24,236 DEBUG [RS:1;jenkins-hbase4:43065] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x51a72024 to 127.0.0.1:63236 2023-07-18 04:54:24,236 DEBUG [RS:1;jenkins-hbase4:43065] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:24,237 INFO [RS:1;jenkins-hbase4:43065] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43065 2023-07-18 04:54:24,245 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43065,1689656057946 2023-07-18 04:54:24,245 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10176c7bf3c0016, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:24,245 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43065,1689656057946 2023-07-18 04:54:24,245 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:24,245 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 04:54:24,247 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43065,1689656057946] 2023-07-18 04:54:24,247 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43065,1689656057946; numProcessing=3 2023-07-18 04:54:24,248 DEBUG [RS:3;jenkins-hbase4:42145] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs 2023-07-18 04:54:24,248 INFO [RS:3;jenkins-hbase4:42145] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C42145%2C1689656058011.jenkins-hbase4.apache.org%2C42145%2C1689656058011.regiongroup-1:(num 1689656059236) 2023-07-18 04:54:24,249 DEBUG [RS:3;jenkins-hbase4:42145] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:24,249 INFO [RS:3;jenkins-hbase4:42145] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:24,249 INFO [RS:3;jenkins-hbase4:42145] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-18 04:54:24,249 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 04:54:24,249 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,43065,1689656057946 already deleted, retry=false 2023-07-18 04:54:24,249 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43065,1689656057946 expired; onlineServers=1 2023-07-18 04:54:24,250 INFO [RS:3;jenkins-hbase4:42145] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42145 2023-07-18 04:54:24,253 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,42145,1689656058011 2023-07-18 04:54:24,253 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10176c7bf3c0016, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:24,253 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:24,253 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 04:54:24,254 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42145,1689656058011] 2023-07-18 04:54:24,254 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42145,1689656058011; numProcessing=4 2023-07-18 04:54:24,255 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,42145,1689656058011 already deleted, retry=false 2023-07-18 04:54:24,255 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42145,1689656058011 expired; onlineServers=0 2023-07-18 04:54:24,255 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34379,1689656057870' ***** 2023-07-18 04:54:24,256 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-18 04:54:24,257 DEBUG [M:0;jenkins-hbase4:34379] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@58078b2e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 04:54:24,257 INFO [M:0;jenkins-hbase4:34379] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34379,1689656057870 2023-07-18 04:54:24,257 INFO [M:0;jenkins-hbase4:34379] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34379,1689656057870; all regions closed. 2023-07-18 04:54:24,257 DEBUG [M:0;jenkins-hbase4:34379] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:24,257 DEBUG [M:0;jenkins-hbase4:34379] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-18 04:54:24,257 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-18 04:54:24,257 DEBUG [M:0;jenkins-hbase4:34379] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-18 04:54:24,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656058265] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656058265,5,FailOnTimeoutGroup] 2023-07-18 04:54:24,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656058265] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656058265,5,FailOnTimeoutGroup] 2023-07-18 04:54:24,258 INFO [M:0;jenkins-hbase4:34379] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-18 04:54:24,260 INFO [M:0;jenkins-hbase4:34379] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-18 04:54:24,260 INFO [M:0;jenkins-hbase4:34379] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-18 04:54:24,260 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-18 04:54:24,260 DEBUG [M:0;jenkins-hbase4:34379] master.HMaster(1512): Stopping service threads 2023-07-18 04:54:24,260 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:24,260 INFO [M:0;jenkins-hbase4:34379] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-18 04:54:24,261 ERROR [M:0;jenkins-hbase4:34379] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-07-18 04:54:24,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 04:54:24,261 INFO [M:0;jenkins-hbase4:34379] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-18 04:54:24,262 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-18 04:54:24,262 DEBUG [M:0;jenkins-hbase4:34379] zookeeper.ZKUtil(398): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-18 04:54:24,262 WARN [M:0;jenkins-hbase4:34379] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-18 04:54:24,262 INFO [M:0;jenkins-hbase4:34379] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-18 04:54:24,263 INFO [M:0;jenkins-hbase4:34379] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-18 04:54:24,264 DEBUG [M:0;jenkins-hbase4:34379] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-18 04:54:24,264 INFO [M:0;jenkins-hbase4:34379] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:24,264 DEBUG [M:0;jenkins-hbase4:34379] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:24,264 DEBUG [M:0;jenkins-hbase4:34379] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-18 04:54:24,264 DEBUG [M:0;jenkins-hbase4:34379] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:24,264 INFO [M:0;jenkins-hbase4:34379] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=55.40 KB heapSize=65.18 KB 2023-07-18 04:54:24,268 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:24,270 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:24,280 INFO [M:0;jenkins-hbase4:34379] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=55.40 KB at sequenceid=116 (bloomFilter=true), to=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/2bdbc267c2bb4e088935a47685bc6cbe 2023-07-18 04:54:24,287 DEBUG [M:0;jenkins-hbase4:34379] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/2bdbc267c2bb4e088935a47685bc6cbe as hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2bdbc267c2bb4e088935a47685bc6cbe 2023-07-18 04:54:24,293 INFO [M:0;jenkins-hbase4:34379] regionserver.HStore(1080): Added hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2bdbc267c2bb4e088935a47685bc6cbe, entries=14, sequenceid=116, filesize=8.1 K 2023-07-18 04:54:24,295 INFO [M:0;jenkins-hbase4:34379] regionserver.HRegion(2948): Finished flush of dataSize ~55.40 KB/56732, heapSize ~65.16 KB/66728, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 30ms, sequenceid=116, compaction requested=false 2023-07-18 04:54:24,297 INFO [M:0;jenkins-hbase4:34379] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:24,297 DEBUG [M:0;jenkins-hbase4:34379] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 04:54:24,297 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,34379,1689656057870 2023-07-18 04:54:24,303 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 04:54:24,303 INFO [M:0;jenkins-hbase4:34379] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-18 04:54:24,304 INFO [M:0;jenkins-hbase4:34379] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34379 2023-07-18 04:54:24,305 DEBUG [M:0;jenkins-hbase4:34379] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,34379,1689656057870 already deleted, retry=false 2023-07-18 04:54:24,334 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:24,375 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:24,375 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:24,441 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:24,481 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:24,481 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:24,501 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:24,501 INFO [M:0;jenkins-hbase4:34379] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34379,1689656057870; zookeeper connection closed. 2023-07-18 04:54:24,501 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:34379-0x10176c7bf3c0007, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:24,548 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:24,587 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:24,587 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:24,601 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:24,601 INFO [RS:3;jenkins-hbase4:42145] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42145,1689656058011; zookeeper connection closed. 2023-07-18 04:54:24,601 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:42145-0x10176c7bf3c000b, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:24,603 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6f231912] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6f231912 2023-07-18 04:54:24,653 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:24,693 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:24,693 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:24,702 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:24,702 INFO [RS:1;jenkins-hbase4:43065] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43065,1689656057946; zookeeper connection closed. 2023-07-18 04:54:24,702 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43065-0x10176c7bf3c0009, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:24,703 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5bf72bb9] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5bf72bb9 2023-07-18 04:54:24,759 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:24,799 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:24,799 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:24,865 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:24,902 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:24,902 INFO [RS:0;jenkins-hbase4:43893] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43893,1689656057910; zookeeper connection closed. 2023-07-18 04:54:24,902 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:43893-0x10176c7bf3c0008, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:24,904 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4528930f] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4528930f 2023-07-18 04:54:24,907 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-18 04:54:24,907 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:24,971 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22269 2023-07-18 04:54:25,002 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:25,002 INFO [RS:2;jenkins-hbase4:36167] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36167,1689656057977; zookeeper connection closed. 2023-07-18 04:54:25,002 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:36167-0x10176c7bf3c000a, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:25,004 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@a46283d] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@a46283d 2023-07-18 04:54:25,005 INFO [Listener at localhost/37011] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 4 regionserver(s) complete 2023-07-18 04:54:25,014 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-07-18 04:54:25,015 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-18 04:54:25,015 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-07-18 04:54:25,015 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/db11400df50c3fb2fe7bab3791162e0a/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0.1689656055500, lastWalPosition=14321, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=false,usedBufferSize=4120] 2023-07-18 04:54:25,015 INFO [SinkThread-7] regionserver.ReplicationSinkManager(110): Current list of sinks is out of date or empty, updating 2023-07-18 04:54:25,015 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:25,016 WARN [SinkThread-7] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 04:54:25,016 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.HBaseInterClusterReplicationEndpoint(575): [Source for peer 1]: Can't replicate because of a local or network error: java.io.IOException: No replication sinks are available at org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:115) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicateEntries(HBaseInterClusterReplicationEndpoint.java:627) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.lambda$createReplicator$3(HBaseInterClusterReplicationEndpoint.java:674) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-18 04:54:25,035 INFO [Listener at localhost/37011] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-18 04:54:25,036 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x06b2f9f8 to 127.0.0.1:63236 2023-07-18 04:54:25,036 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:25,036 DEBUG [Listener at localhost/37011] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-18 04:54:25,036 DEBUG [Listener at localhost/37011] util.JVMClusterUtil(257): Found active master hash=1004921980, stopped=false 2023-07-18 04:54:25,036 INFO [Listener at localhost/37011] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,44113,1689656052898 2023-07-18 04:54:25,039 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-18 04:54:25,039 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:42175-0x10176c7bf3c0001, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-18 04:54:25,039 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-18 04:54:25,039 INFO [Listener at localhost/37011] procedure2.ProcedureExecutor(629): Stopping 2023-07-18 04:54:25,039 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:25,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 04:54:25,039 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x16d9e291 to 127.0.0.1:63236 2023-07-18 04:54:25,041 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:25,041 INFO [Listener at localhost/37011] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42175,1689656054096' ***** 2023-07-18 04:54:25,041 INFO [Listener at localhost/37011] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 04:54:25,041 INFO [Listener at localhost/37011] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45857,1689656054144' ***** 2023-07-18 04:54:25,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 04:54:25,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42175-0x10176c7bf3c0001, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 04:54:25,042 INFO [RS:0;jenkins-hbase4:42175] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 04:54:25,042 INFO [RS:0;jenkins-hbase4:42175] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 04:54:25,042 INFO [RS:0;jenkins-hbase4:42175] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 04:54:25,041 INFO [Listener at localhost/37011] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 04:54:25,042 INFO [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42175,1689656054096 2023-07-18 04:54:25,042 DEBUG [RS:0;jenkins-hbase4:42175] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x18c5f288 to 127.0.0.1:63236 2023-07-18 04:54:25,042 DEBUG [RS:0;jenkins-hbase4:42175] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:25,043 INFO [RS:0;jenkins-hbase4:42175] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 04:54:25,043 INFO [RS:0;jenkins-hbase4:42175] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 04:54:25,043 INFO [RS:0;jenkins-hbase4:42175] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 04:54:25,043 INFO [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-18 04:54:25,042 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 04:54:25,043 INFO [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-18 04:54:25,043 INFO [RS:1;jenkins-hbase4:45857] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 04:54:25,044 DEBUG [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-07-18 04:54:25,044 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 04:54:25,044 DEBUG [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-18 04:54:25,044 INFO [RS:1;jenkins-hbase4:45857] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 04:54:25,045 INFO [RS:1;jenkins-hbase4:45857] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 04:54:25,045 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-18 04:54:25,046 INFO [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(3305): Received CLOSE for 8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:25,046 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-18 04:54:25,046 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-18 04:54:25,046 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-18 04:54:25,046 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-18 04:54:25,046 INFO [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(3305): Received CLOSE for db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:25,046 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.87 KB heapSize=7.44 KB 2023-07-18 04:54:25,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8750ae11f1669b3efa9c8fcc2b1be9cf, disabling compactions & flushes 2023-07-18 04:54:25,046 INFO [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(3305): Received CLOSE for 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:25,047 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:25,047 INFO [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:25,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:25,047 DEBUG [RS:1;jenkins-hbase4:45857] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7a1a433c to 127.0.0.1:63236 2023-07-18 04:54:25,048 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. after waiting 0 ms 2023-07-18 04:54:25,048 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:25,048 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 8750ae11f1669b3efa9c8fcc2b1be9cf 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-18 04:54:25,048 DEBUG [RS:1;jenkins-hbase4:45857] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:25,048 INFO [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-07-18 04:54:25,048 DEBUG [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(1478): Online Regions={8750ae11f1669b3efa9c8fcc2b1be9cf=hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf., db11400df50c3fb2fe7bab3791162e0a=t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a., 7f6a4e55b8779eead03ae75b46b98c60=t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60.} 2023-07-18 04:54:25,048 DEBUG [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(1504): Waiting on 7f6a4e55b8779eead03ae75b46b98c60, 8750ae11f1669b3efa9c8fcc2b1be9cf, db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:25,071 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf/.tmp/info/150be18a83f248079f73a2772765d7c9 2023-07-18 04:54:25,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf/.tmp/info/150be18a83f248079f73a2772765d7c9 as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf/info/150be18a83f248079f73a2772765d7c9 2023-07-18 04:54:25,088 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 29162 2023-07-18 04:54:25,088 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-18 04:54:25,088 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 29162 2023-07-18 04:54:25,088 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1.1689656056513, lastWalPosition=29162, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=false,usedBufferSize=8120] 2023-07-18 04:54:25,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf/info/150be18a83f248079f73a2772765d7c9, entries=2, sequenceid=6, filesize=4.8 K 2023-07-18 04:54:25,094 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 8750ae11f1669b3efa9c8fcc2b1be9cf in 46ms, sequenceid=6, compaction requested=false 2023-07-18 04:54:25,098 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:25,098 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:25,100 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/.tmp/info/97ba85398a9b4013bfdbe13aef0066c8 2023-07-18 04:54:25,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-18 04:54:25,108 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:25,108 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8750ae11f1669b3efa9c8fcc2b1be9cf: 2023-07-18 04:54:25,108 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:25,109 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing db11400df50c3fb2fe7bab3791162e0a, disabling compactions & flushes 2023-07-18 04:54:25,109 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:25,109 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:25,109 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. after waiting 0 ms 2023-07-18 04:54:25,109 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:25,109 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing db11400df50c3fb2fe7bab3791162e0a 2/2 column families, dataSize=6.25 KB heapSize=21.71 KB 2023-07-18 04:54:25,123 INFO [SinkThread-8] regionserver.ReplicationSinkManager(110): Current list of sinks is out of date or empty, updating 2023-07-18 04:54:25,130 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-07-18 04:54:25,130 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 04:54:25,130 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-07-18 04:54:25,130 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:25,135 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.76 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/.tmp/cf1/e88dc907a1be4e2fb34eb8d7b6bd8b66 2023-07-18 04:54:25,142 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e88dc907a1be4e2fb34eb8d7b6bd8b66 2023-07-18 04:54:25,143 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/.tmp/rep_barrier/d9ffd0a2214e4da78f73687fcca695d9 2023-07-18 04:54:25,152 WARN [SinkThread-8] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 04:54:25,152 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.HBaseInterClusterReplicationEndpoint(575): [Source for peer 1]: Can't replicate because of a local or network error: java.io.IOException: No replication sinks are available at org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:115) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicateEntries(HBaseInterClusterReplicationEndpoint.java:627) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.lambda$createReplicator$3(HBaseInterClusterReplicationEndpoint.java:674) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-18 04:54:25,171 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.50 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/.tmp/norep/6cf2207f2f674243be9f1b7a2f68549b 2023-07-18 04:54:25,177 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/.tmp/table/cb85068bae134b3cb32bf67c6cb9911e 2023-07-18 04:54:25,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6cf2207f2f674243be9f1b7a2f68549b 2023-07-18 04:54:25,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/.tmp/cf1/e88dc907a1be4e2fb34eb8d7b6bd8b66 as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/cf1/e88dc907a1be4e2fb34eb8d7b6bd8b66 2023-07-18 04:54:25,186 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/.tmp/info/97ba85398a9b4013bfdbe13aef0066c8 as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/info/97ba85398a9b4013bfdbe13aef0066c8 2023-07-18 04:54:25,186 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e88dc907a1be4e2fb34eb8d7b6bd8b66 2023-07-18 04:54:25,186 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/cf1/e88dc907a1be4e2fb34eb8d7b6bd8b66, entries=100, sequenceid=106, filesize=8.2 K 2023-07-18 04:54:25,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/.tmp/norep/6cf2207f2f674243be9f1b7a2f68549b as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/norep/6cf2207f2f674243be9f1b7a2f68549b 2023-07-18 04:54:25,194 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/info/97ba85398a9b4013bfdbe13aef0066c8, entries=30, sequenceid=19, filesize=8.1 K 2023-07-18 04:54:25,196 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/.tmp/rep_barrier/d9ffd0a2214e4da78f73687fcca695d9 as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/rep_barrier/d9ffd0a2214e4da78f73687fcca695d9 2023-07-18 04:54:25,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6cf2207f2f674243be9f1b7a2f68549b 2023-07-18 04:54:25,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/norep/6cf2207f2f674243be9f1b7a2f68549b, entries=51, sequenceid=106, filesize=6.5 K 2023-07-18 04:54:25,197 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 29747 2023-07-18 04:54:25,197 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 04:54:25,197 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 29747 2023-07-18 04:54:25,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~6.25 KB/6401, heapSize ~21.68 KB/22200, currentSize=0 B/0 for db11400df50c3fb2fe7bab3791162e0a in 89ms, sequenceid=106, compaction requested=false 2023-07-18 04:54:25,209 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/rep_barrier/d9ffd0a2214e4da78f73687fcca695d9, entries=2, sequenceid=19, filesize=4.9 K 2023-07-18 04:54:25,211 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/.tmp/table/cb85068bae134b3cb32bf67c6cb9911e as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/table/cb85068bae134b3cb32bf67c6cb9911e 2023-07-18 04:54:25,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/recovered.edits/109.seqid, newMaxSeqId=109, maxSeqId=1 2023-07-18 04:54:25,213 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:25,213 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for db11400df50c3fb2fe7bab3791162e0a: 2023-07-18 04:54:25,213 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:25,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7f6a4e55b8779eead03ae75b46b98c60, disabling compactions & flushes 2023-07-18 04:54:25,214 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:25,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:25,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. after waiting 0 ms 2023-07-18 04:54:25,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:25,214 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 7f6a4e55b8779eead03ae75b46b98c60 2/2 column families, dataSize=12.70 KB heapSize=42.80 KB 2023-07-18 04:54:25,220 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/table/cb85068bae134b3cb32bf67c6cb9911e, entries=6, sequenceid=19, filesize=4.8 K 2023-07-18 04:54:25,225 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.87 KB/3964, heapSize ~7.39 KB/7568, currentSize=0 B/0 for 1588230740 in 179ms, sequenceid=19, compaction requested=false 2023-07-18 04:54:25,243 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-07-18 04:54:25,243 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:25,243 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 04:54:25,245 DEBUG [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-18 04:54:25,248 DEBUG [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(1504): Waiting on 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:25,256 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.74 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/.tmp/cf1/6e0003805a0f4f4c884b382a37632a83 2023-07-18 04:54:25,258 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-18 04:54:25,259 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-18 04:54:25,260 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-18 04:54:25,260 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-18 04:54:25,260 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-18 04:54:25,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6e0003805a0f4f4c884b382a37632a83 2023-07-18 04:54:25,277 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.96 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/.tmp/norep/27c15cac5541443d89a7c280b70cebb3 2023-07-18 04:54:25,282 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27c15cac5541443d89a7c280b70cebb3 2023-07-18 04:54:25,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/.tmp/cf1/6e0003805a0f4f4c884b382a37632a83 as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/cf1/6e0003805a0f4f4c884b382a37632a83 2023-07-18 04:54:25,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6e0003805a0f4f4c884b382a37632a83 2023-07-18 04:54:25,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/cf1/6e0003805a0f4f4c884b382a37632a83, entries=200, sequenceid=206, filesize=11.9 K 2023-07-18 04:54:25,291 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/.tmp/norep/27c15cac5541443d89a7c280b70cebb3 as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/norep/27c15cac5541443d89a7c280b70cebb3 2023-07-18 04:54:25,298 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27c15cac5541443d89a7c280b70cebb3 2023-07-18 04:54:25,298 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/norep/27c15cac5541443d89a7c280b70cebb3, entries=101, sequenceid=206, filesize=8.3 K 2023-07-18 04:54:25,299 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~12.70 KB/13001, heapSize ~42.77 KB/43800, currentSize=0 B/0 for 7f6a4e55b8779eead03ae75b46b98c60 in 85ms, sequenceid=206, compaction requested=false 2023-07-18 04:54:25,311 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 30301 2023-07-18 04:54:25,311 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 04:54:25,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/recovered.edits/209.seqid, newMaxSeqId=209, maxSeqId=1 2023-07-18 04:54:25,314 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:25,315 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7f6a4e55b8779eead03ae75b46b98c60: 2023-07-18 04:54:25,315 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:25,339 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-18 04:54:25,339 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-18 04:54:25,339 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-18 04:54:25,339 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-18 04:54:25,351 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:25,356 INFO [SinkThread-9] regionserver.ReplicationSinkManager(110): Current list of sinks is out of date or empty, updating 2023-07-18 04:54:25,357 WARN [SinkThread-9] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 04:54:25,357 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.HBaseInterClusterReplicationEndpoint(575): [Source for peer 1]: Can't replicate because of a local or network error: java.io.IOException: No replication sinks are available at org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:115) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicateEntries(HBaseInterClusterReplicationEndpoint.java:627) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.lambda$createReplicator$3(HBaseInterClusterReplicationEndpoint.java:674) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-18 04:54:25,376 INFO [jenkins-hbase4:42175Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0]: currently replicating from: hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096/jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0.1689656055499 at position: -1 2023-07-18 04:54:25,376 INFO [jenkins-hbase4:45857Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=12240B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 300, current progress: walGroup [jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1]: currently replicating from: hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1.1689656056513 at position: 22269 walGroup [jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0]: currently replicating from: hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0.1689656055500 at position: 10975 2023-07-18 04:54:25,445 INFO [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42175,1689656054096; all regions closed. 2023-07-18 04:54:25,445 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096 2023-07-18 04:54:25,449 INFO [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45857,1689656054144; all regions closed. 2023-07-18 04:54:25,450 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:25,456 DEBUG [RS:0;jenkins-hbase4:42175] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs 2023-07-18 04:54:25,457 INFO [RS:0;jenkins-hbase4:42175] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C42175%2C1689656054096.meta:.meta(num 1689656055751) 2023-07-18 04:54:25,458 DEBUG [RS:1;jenkins-hbase4:45857] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs 2023-07-18 04:54:25,458 INFO [RS:1;jenkins-hbase4:45857] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1:(num 1689656056513) 2023-07-18 04:54:25,458 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096 2023-07-18 04:54:25,458 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:25,471 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:25,473 DEBUG [RS:0;jenkins-hbase4:42175] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs 2023-07-18 04:54:25,474 INFO [RS:0;jenkins-hbase4:42175] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0:(num 1689656055499) 2023-07-18 04:54:25,474 DEBUG [RS:0;jenkins-hbase4:42175] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:25,474 INFO [RS:0;jenkins-hbase4:42175] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:25,474 INFO [RS:0;jenkins-hbase4:42175] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-18 04:54:25,474 INFO [RS:0;jenkins-hbase4:42175] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-18 04:54:25,474 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 04:54:25,478 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] regionserver.WALEntryStream(207): Couldn't get file length information about log hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096/jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0.1689656055499, it was closed cleanly currently replicating from: hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096/jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0.1689656055499 at position: 0 2023-07-18 04:54:25,478 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096/jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0.1689656055499 2023-07-18 04:54:25,478 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 04:54:25,478 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096/jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0.1689656055499, lastWalPosition=-1, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-07-18 04:54:25,480 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0.1689656055499] 2023-07-18 04:54:25,605 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10176c7bf3c0018, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:25,605 DEBUG [RS:0;jenkins-hbase4:42175] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6c5e2a11 to 127.0.0.1:63236 2023-07-18 04:54:25,608 DEBUG [RS:0;jenkins-hbase4:42175] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:25,608 INFO [RS:0;jenkins-hbase4:42175] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C42175%2C1689656054096.jenkins-hbase4.apache.org%2C42175%2C1689656054096.regiongroup-0,1 terminated 2023-07-18 04:54:25,609 INFO [RS:0;jenkins-hbase4:42175] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42175 2023-07-18 04:54:25,611 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:42175-0x10176c7bf3c0001, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,42175,1689656054096 2023-07-18 04:54:25,611 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 04:54:25,611 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:42175-0x10176c7bf3c0001, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 04:54:25,611 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,42175,1689656054096 2023-07-18 04:54:25,612 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 04:54:25,613 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42175,1689656054096] 2023-07-18 04:54:25,613 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42175,1689656054096; numProcessing=1 2023-07-18 04:54:25,614 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,42175,1689656054096 already deleted, retry=false 2023-07-18 04:54:25,615 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42175,1689656054096 expired; onlineServers=1 2023-07-18 04:54:25,658 INFO [SinkThread-0] regionserver.ReplicationSinkManager(110): Current list of sinks is out of date or empty, updating 2023-07-18 04:54:25,659 WARN [SinkThread-0] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 04:54:25,659 WARN [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1] regionserver.HBaseInterClusterReplicationEndpoint(575): [Source for peer 1]: Can't replicate because of a local or network error: java.io.IOException: No replication sinks are available at org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:115) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicateEntries(HBaseInterClusterReplicationEndpoint.java:627) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.lambda$createReplicator$3(HBaseInterClusterReplicationEndpoint.java:674) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-18 04:54:25,811 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:42175-0x10176c7bf3c0001, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:25,811 INFO [RS:0;jenkins-hbase4:42175] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42175,1689656054096; zookeeper connection closed. 2023-07-18 04:54:25,811 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:42175-0x10176c7bf3c0001, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:25,811 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@63a011d1] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@63a011d1 2023-07-18 04:54:25,874 DEBUG [RS:1;jenkins-hbase4:45857] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs 2023-07-18 04:54:25,874 INFO [RS:1;jenkins-hbase4:45857] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0:(num 1689656055500) 2023-07-18 04:54:25,874 DEBUG [RS:1;jenkins-hbase4:45857] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:25,874 INFO [RS:1;jenkins-hbase4:45857] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:25,875 INFO [RS:1;jenkins-hbase4:45857] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-18 04:54:25,875 INFO [RS:1;jenkins-hbase4:45857] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 04:54:25,875 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 04:54:25,875 INFO [RS:1;jenkins-hbase4:45857] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 04:54:25,875 INFO [RS:1;jenkins-hbase4:45857] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 04:54:25,875 INFO [RS:1;jenkins-hbase4:45857] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-18 04:54:25,978 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10176c7bf3c0016, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:25,978 DEBUG [RS:1;jenkins-hbase4:45857] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6f23048d to 127.0.0.1:63236 2023-07-18 04:54:25,978 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10176c7bf3c0016, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:25,978 DEBUG [RS:1;jenkins-hbase4:45857] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:26,179 INFO [RS:1;jenkins-hbase4:45857] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1 terminated 2023-07-18 04:54:26,180 INFO [RS:1;jenkins-hbase4:45857] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1 terminated 2023-07-18 04:54:26,180 INFO [RS:1;jenkins-hbase4:45857] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45857 2023-07-18 04:54:26,184 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 04:54:26,184 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:26,185 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45857,1689656054144] 2023-07-18 04:54:26,185 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45857,1689656054144; numProcessing=2 2023-07-18 04:54:26,186 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,45857,1689656054144 already deleted, retry=false 2023-07-18 04:54:26,186 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45857,1689656054144 expired; onlineServers=0 2023-07-18 04:54:26,186 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,44113,1689656052898' ***** 2023-07-18 04:54:26,186 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-18 04:54:26,186 DEBUG [M:0;jenkins-hbase4:44113] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7ad8ee47, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 04:54:26,187 INFO [M:0;jenkins-hbase4:44113] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,44113,1689656052898 2023-07-18 04:54:26,187 INFO [M:0;jenkins-hbase4:44113] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,44113,1689656052898; all regions closed. 2023-07-18 04:54:26,187 DEBUG [M:0;jenkins-hbase4:44113] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:26,187 DEBUG [M:0;jenkins-hbase4:44113] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-18 04:54:26,187 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-18 04:54:26,187 DEBUG [M:0;jenkins-hbase4:44113] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-18 04:54:26,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656055167] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656055167,5,FailOnTimeoutGroup] 2023-07-18 04:54:26,187 INFO [M:0;jenkins-hbase4:44113] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-18 04:54:26,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656055166] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656055166,5,FailOnTimeoutGroup] 2023-07-18 04:54:26,188 INFO [M:0;jenkins-hbase4:44113] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-18 04:54:26,188 INFO [M:0;jenkins-hbase4:44113] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-18 04:54:26,189 DEBUG [M:0;jenkins-hbase4:44113] master.HMaster(1512): Stopping service threads 2023-07-18 04:54:26,189 INFO [M:0;jenkins-hbase4:44113] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-18 04:54:26,189 ERROR [M:0;jenkins-hbase4:44113] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] 2023-07-18 04:54:26,189 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-18 04:54:26,189 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:26,189 INFO [M:0;jenkins-hbase4:44113] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-18 04:54:26,189 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-18 04:54:26,190 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-18 04:54:26,190 DEBUG [M:0;jenkins-hbase4:44113] zookeeper.ZKUtil(398): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-18 04:54:26,190 WARN [M:0;jenkins-hbase4:44113] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-18 04:54:26,190 INFO [M:0;jenkins-hbase4:44113] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-18 04:54:26,190 INFO [M:0;jenkins-hbase4:44113] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-18 04:54:26,191 DEBUG [M:0;jenkins-hbase4:44113] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-18 04:54:26,191 INFO [M:0;jenkins-hbase4:44113] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:26,191 DEBUG [M:0;jenkins-hbase4:44113] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:26,191 DEBUG [M:0;jenkins-hbase4:44113] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-18 04:54:26,191 DEBUG [M:0;jenkins-hbase4:44113] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:26,191 INFO [M:0;jenkins-hbase4:44113] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=61.04 KB heapSize=72.60 KB 2023-07-18 04:54:26,205 INFO [M:0;jenkins-hbase4:44113] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=61.04 KB at sequenceid=136 (bloomFilter=true), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3e1e56add1154f2eb5422b15934f49bc 2023-07-18 04:54:26,211 DEBUG [M:0;jenkins-hbase4:44113] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3e1e56add1154f2eb5422b15934f49bc as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3e1e56add1154f2eb5422b15934f49bc 2023-07-18 04:54:26,216 INFO [M:0;jenkins-hbase4:44113] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3e1e56add1154f2eb5422b15934f49bc, entries=17, sequenceid=136, filesize=8.5 K 2023-07-18 04:54:26,218 INFO [M:0;jenkins-hbase4:44113] regionserver.HRegion(2948): Finished flush of dataSize ~61.04 KB/62508, heapSize ~72.59 KB/74328, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 27ms, sequenceid=136, compaction requested=false 2023-07-18 04:54:26,220 INFO [M:0;jenkins-hbase4:44113] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:26,220 DEBUG [M:0;jenkins-hbase4:44113] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 04:54:26,220 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/WALs/jenkins-hbase4.apache.org,44113,1689656052898 2023-07-18 04:54:26,223 INFO [M:0;jenkins-hbase4:44113] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-18 04:54:26,223 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 04:54:26,224 INFO [M:0;jenkins-hbase4:44113] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44113 2023-07-18 04:54:26,227 DEBUG [M:0;jenkins-hbase4:44113] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,44113,1689656052898 already deleted, retry=false 2023-07-18 04:54:26,285 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:26,285 INFO [RS:1;jenkins-hbase4:45857] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45857,1689656054144; zookeeper connection closed. 2023-07-18 04:54:26,285 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): regionserver:45857-0x10176c7bf3c0002, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:26,285 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3a69b93e] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3a69b93e 2023-07-18 04:54:26,286 INFO [Listener at localhost/37011] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 2 regionserver(s) complete 2023-07-18 04:54:26,385 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:26,385 INFO [M:0;jenkins-hbase4:44113] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,44113,1689656052898; zookeeper connection closed. 2023-07-18 04:54:26,385 DEBUG [Listener at localhost/46199-EventThread] zookeeper.ZKWatcher(600): master:44113-0x10176c7bf3c0000, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:26,388 INFO [Listener at localhost/37011] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 04:54:26,388 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:26,388 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:26,388 INFO [Listener at localhost/37011] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 04:54:26,388 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:26,388 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 04:54:26,389 INFO [Listener at localhost/37011] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 04:54:26,390 INFO [Listener at localhost/37011] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33929 2023-07-18 04:54:26,391 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:26,392 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:26,393 INFO [Listener at localhost/37011] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33929 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:26,396 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:339290x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:26,397 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:33929-0x10176c7bf3c001c connected 2023-07-18 04:54:26,400 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 04:54:26,400 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:26,401 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 04:54:26,401 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33929 2023-07-18 04:54:26,401 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33929 2023-07-18 04:54:26,402 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33929 2023-07-18 04:54:26,402 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33929 2023-07-18 04:54:26,402 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33929 2023-07-18 04:54:26,402 INFO [Listener at localhost/37011] master.HMaster(444): hbase.rootdir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0, hbase.cluster.distributed=false 2023-07-18 04:54:26,404 DEBUG [pool-448-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-18 04:54:26,415 INFO [Listener at localhost/37011] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 04:54:26,416 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:26,416 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:26,416 INFO [Listener at localhost/37011] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 04:54:26,416 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:26,416 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 04:54:26,416 INFO [Listener at localhost/37011] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 04:54:26,418 INFO [Listener at localhost/37011] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37265 2023-07-18 04:54:26,418 INFO [Listener at localhost/37011] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 04:54:26,419 DEBUG [Listener at localhost/37011] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 04:54:26,420 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:26,421 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:26,422 INFO [Listener at localhost/37011] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37265 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:26,427 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:372650x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:26,429 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:372650x0, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 04:54:26,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:37265-0x10176c7bf3c001d connected 2023-07-18 04:54:26,429 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:37265-0x10176c7bf3c001d, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:26,430 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:37265-0x10176c7bf3c001d, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 04:54:26,430 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37265 2023-07-18 04:54:26,430 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37265 2023-07-18 04:54:26,430 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37265 2023-07-18 04:54:26,434 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37265 2023-07-18 04:54:26,435 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37265 2023-07-18 04:54:26,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,33929,1689656066387 2023-07-18 04:54:26,437 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-18 04:54:26,438 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,33929,1689656066387 2023-07-18 04:54:26,439 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:37265-0x10176c7bf3c001d, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 04:54:26,439 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 04:54:26,439 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:26,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-18 04:54:26,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,33929,1689656066387 from backup master directory 2023-07-18 04:54:26,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-18 04:54:26,442 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,33929,1689656066387 2023-07-18 04:54:26,442 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 04:54:26,442 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-18 04:54:26,442 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,33929,1689656066387 2023-07-18 04:54:26,458 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:26,486 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x60b3ad3e to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:26,491 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@c625a37, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:26,491 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 04:54:26,492 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-18 04:54:26,492 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:26,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:26,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,34379,1689656057870 to hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,34379,1689656057870-dead as it is dead 2023-07-18 04:54:26,501 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,34379,1689656057870-dead/jenkins-hbase4.apache.org%2C34379%2C1689656057870.jenkins-hbase4.apache.org%2C34379%2C1689656057870.regiongroup-0.1689656058165 2023-07-18 04:54:26,505 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,34379,1689656057870-dead/jenkins-hbase4.apache.org%2C34379%2C1689656057870.jenkins-hbase4.apache.org%2C34379%2C1689656057870.regiongroup-0.1689656058165 after 4ms 2023-07-18 04:54:26,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,34379,1689656057870-dead/jenkins-hbase4.apache.org%2C34379%2C1689656057870.jenkins-hbase4.apache.org%2C34379%2C1689656057870.regiongroup-0.1689656058165 to hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34379%2C1689656057870.jenkins-hbase4.apache.org%2C34379%2C1689656057870.regiongroup-0.1689656058165 2023-07-18 04:54:26,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,34379,1689656057870-dead 2023-07-18 04:54:26,506 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,33929,1689656066387 2023-07-18 04:54:26,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:26,509 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33929%2C1689656066387.jenkins-hbase4.apache.org%2C33929%2C1689656066387.regiongroup-0, suffix=, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,33929,1689656066387, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/oldWALs, maxLogs=10 2023-07-18 04:54:26,515 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,33929,1689656066387/jenkins-hbase4.apache.org%2C33929%2C1689656066387.jenkins-hbase4.apache.org%2C33929%2C1689656066387.regiongroup-0.1689656066509 2023-07-18 04:54:26,515 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40889,DS-cb62bdfa-114c-4e1c-89f3-b40b9c3afb83,DISK], DatanodeInfoWithStorage[127.0.0.1:37603,DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347,DISK], DatanodeInfoWithStorage[127.0.0.1:38949,DS-7e9427a5-d2d9-4ece-8da8-f6fb77b8864e,DISK]] 2023-07-18 04:54:26,515 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:26,515 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:26,516 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:26,516 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:26,517 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:26,518 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-18 04:54:26,518 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-18 04:54:26,524 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2bdbc267c2bb4e088935a47685bc6cbe 2023-07-18 04:54:26,525 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:26,526 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-18 04:54:26,526 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34379%2C1689656057870.jenkins-hbase4.apache.org%2C34379%2C1689656057870.regiongroup-0.1689656058165 2023-07-18 04:54:26,532 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 131, firstSequenceIdInLog=3, maxSequenceIdInLog=118, path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34379%2C1689656057870.jenkins-hbase4.apache.org%2C34379%2C1689656057870.regiongroup-0.1689656058165 2023-07-18 04:54:26,533 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34379%2C1689656057870.jenkins-hbase4.apache.org%2C34379%2C1689656057870.regiongroup-0.1689656058165 2023-07-18 04:54:26,536 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:26,538 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=1 2023-07-18 04:54:26,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=119; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59706187, jitterRate=-0.11030848324298859}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 04:54:26,539 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 04:54:26,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-18 04:54:26,541 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-18 04:54:26,541 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-18 04:54:26,541 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-18 04:54:26,542 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-18 04:54:26,557 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-18 04:54:26,558 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-18 04:54:26,558 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-18 04:54:26,558 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-18 04:54:26,559 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-18 04:54:26,559 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-18 04:54:26,560 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 17 msec 2023-07-18 04:54:26,560 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-18 04:54:26,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-18 04:54:26,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42145,1689656058011, table=hbase:meta, region=1588230740 2023-07-18 04:54:26,563 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 4 possibly 'live' servers, and 0 'splitting'. 2023-07-18 04:54:26,565 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,43065,1689656057946 already deleted, retry=false 2023-07-18 04:54:26,565 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,43065,1689656057946 on jenkins-hbase4.apache.org,33929,1689656066387 2023-07-18 04:54:26,577 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,43065,1689656057946, splitWal=true, meta=false 2023-07-18 04:54:26,578 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=15 for jenkins-hbase4.apache.org,43065,1689656057946 (carryingMeta=false) jenkins-hbase4.apache.org,43065,1689656057946/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@65cdd4ce[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-18 04:54:26,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,36167,1689656057977 already deleted, retry=false 2023-07-18 04:54:26,580 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,36167,1689656057977 on jenkins-hbase4.apache.org,33929,1689656066387 2023-07-18 04:54:26,581 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=16, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,36167,1689656057977, splitWal=true, meta=false 2023-07-18 04:54:26,581 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=16 for jenkins-hbase4.apache.org,36167,1689656057977 (carryingMeta=false) jenkins-hbase4.apache.org,36167,1689656057977/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@761feef6[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-18 04:54:26,583 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,42145,1689656058011 already deleted, retry=false 2023-07-18 04:54:26,583 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,42145,1689656058011 on jenkins-hbase4.apache.org,33929,1689656066387 2023-07-18 04:54:26,584 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,42145,1689656058011, splitWal=true, meta=true 2023-07-18 04:54:26,585 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=17 for jenkins-hbase4.apache.org,42145,1689656058011 (carryingMeta=true) jenkins-hbase4.apache.org,42145,1689656058011/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@7dff3f5f[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-18 04:54:26,586 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,43893,1689656057910 already deleted, retry=false 2023-07-18 04:54:26,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,43893,1689656057910 on jenkins-hbase4.apache.org,33929,1689656066387 2023-07-18 04:54:26,587 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,43893,1689656057910, splitWal=true, meta=false 2023-07-18 04:54:26,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,43893,1689656057910 (carryingMeta=false) jenkins-hbase4.apache.org,43893,1689656057910/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@449c8e75[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-18 04:54:26,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-18 04:54:26,598 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-18 04:54:26,598 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-18 04:54:26,598 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-18 04:54:26,599 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-18 04:54:26,599 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-18 04:54:26,600 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-18 04:54:26,600 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-18 04:54:26,602 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:37265-0x10176c7bf3c001d, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 04:54:26,602 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 04:54:26,602 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:26,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,33929,1689656066387, sessionid=0x10176c7bf3c001c, setting cluster-up flag (Was=false) 2023-07-18 04:54:26,607 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-18 04:54:26,608 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33929,1689656066387 2023-07-18 04:54:26,611 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-18 04:54:26,612 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33929,1689656066387 2023-07-18 04:54:26,612 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.hbase-snapshot/.tmp 2023-07-18 04:54:26,613 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:26,613 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:26,613 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:26,613 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:26,613 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-18 04:54:26,613 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:26,614 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 04:54:26,614 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:26,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689656096619 2023-07-18 04:54:26,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-18 04:54:26,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-18 04:54:26,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-18 04:54:26,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-18 04:54:26,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-18 04:54:26,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-18 04:54:26,622 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:26,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-18 04:54:26,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-18 04:54:26,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-18 04:54:26,624 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-18 04:54:26,624 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-18 04:54:26,624 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656066624,5,FailOnTimeoutGroup] 2023-07-18 04:54:26,624 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42145,1689656058011; numProcessing=1 2023-07-18 04:54:26,625 DEBUG [PEWorker-5] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43893,1689656057910; numProcessing=2 2023-07-18 04:54:26,625 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43065,1689656057946; numProcessing=3 2023-07-18 04:54:26,625 DEBUG [PEWorker-4] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36167,1689656057977; numProcessing=4 2023-07-18 04:54:26,625 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=17, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42145,1689656058011, splitWal=true, meta=true 2023-07-18 04:54:26,625 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=15, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43065,1689656057946, splitWal=true, meta=false 2023-07-18 04:54:26,625 INFO [PEWorker-4] procedure.ServerCrashProcedure(161): Start pid=16, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,36167,1689656057977, splitWal=true, meta=false 2023-07-18 04:54:26,625 INFO [PEWorker-5] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43893,1689656057910, splitWal=true, meta=false 2023-07-18 04:54:26,626 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=17, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42145,1689656058011, splitWal=true, meta=true, isMeta: true 2023-07-18 04:54:26,626 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656066624,5,FailOnTimeoutGroup] 2023-07-18 04:54:26,626 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:26,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-18 04:54:26,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:26,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:26,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689656066627, completionTime=-1 2023-07-18 04:54:26,627 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-18 04:54:26,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-18 04:54:26,632 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011-splitting 2023-07-18 04:54:26,633 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 04:54:26,633 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011-splitting dir is empty, no logs to split. 2023-07-18 04:54:26,634 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,42145,1689656058011 WAL count=0, meta=true 2023-07-18 04:54:26,639 INFO [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(951): ClusterId : b492e8cd-f4f9-4591-97c6-7f2edd4114a8 2023-07-18 04:54:26,639 DEBUG [RS:0;jenkins-hbase4:37265] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 04:54:26,641 DEBUG [RS:0;jenkins-hbase4:37265] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 04:54:26,641 DEBUG [RS:0;jenkins-hbase4:37265] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 04:54:26,644 DEBUG [RS:0;jenkins-hbase4:37265] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 04:54:26,646 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011-splitting dir is empty, no logs to split. 2023-07-18 04:54:26,646 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,42145,1689656058011 WAL count=0, meta=true 2023-07-18 04:54:26,646 DEBUG [RS:0;jenkins-hbase4:37265] zookeeper.ReadOnlyZKClient(139): Connect 0x0c3bf1b5 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:26,646 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,42145,1689656058011 WAL splitting is done? wals=0, meta=true 2023-07-18 04:54:26,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-18 04:54:26,651 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=19, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-18 04:54:26,653 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=19, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 04:54:26,663 DEBUG [RS:0;jenkins-hbase4:37265] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2fc1e42, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:26,663 DEBUG [RS:0;jenkins-hbase4:37265] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c39ac5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 04:54:26,673 DEBUG [RS:0;jenkins-hbase4:37265] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:37265 2023-07-18 04:54:26,673 INFO [RS:0;jenkins-hbase4:37265] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 04:54:26,673 INFO [RS:0;jenkins-hbase4:37265] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 04:54:26,673 DEBUG [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 04:54:26,674 INFO [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33929,1689656066387 with isa=jenkins-hbase4.apache.org/172.31.14.131:37265, startcode=1689656066415 2023-07-18 04:54:26,674 DEBUG [RS:0;jenkins-hbase4:37265] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 04:54:26,682 INFO [RS-EventLoopGroup-11-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42733, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 04:54:26,683 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33929] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:26,683 DEBUG [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0 2023-07-18 04:54:26,684 DEBUG [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37939 2023-07-18 04:54:26,684 DEBUG [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 04:54:26,690 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:26,691 DEBUG [RS:0;jenkins-hbase4:37265] zookeeper.ZKUtil(162): regionserver:37265-0x10176c7bf3c001d, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:26,692 WARN [RS:0;jenkins-hbase4:37265] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 04:54:26,692 INFO [RS:0;jenkins-hbase4:37265] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:26,692 INFO [RS:0;jenkins-hbase4:37265] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:26,692 DEBUG [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:26,693 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37265,1689656066415] 2023-07-18 04:54:26,698 DEBUG [RS:0;jenkins-hbase4:37265] zookeeper.ZKUtil(162): regionserver:37265-0x10176c7bf3c001d, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:26,699 DEBUG [RS:0;jenkins-hbase4:37265] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 04:54:26,699 INFO [RS:0;jenkins-hbase4:37265] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 04:54:26,720 INFO [RS:0;jenkins-hbase4:37265] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 04:54:26,720 INFO [RS:0;jenkins-hbase4:37265] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 04:54:26,720 INFO [RS:0;jenkins-hbase4:37265] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:26,721 INFO [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 04:54:26,723 INFO [RS:0;jenkins-hbase4:37265] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:26,723 DEBUG [RS:0;jenkins-hbase4:37265] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:26,723 DEBUG [RS:0;jenkins-hbase4:37265] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:26,723 DEBUG [RS:0;jenkins-hbase4:37265] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:26,723 DEBUG [RS:0;jenkins-hbase4:37265] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:26,723 DEBUG [RS:0;jenkins-hbase4:37265] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:26,723 DEBUG [RS:0;jenkins-hbase4:37265] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 04:54:26,723 DEBUG [RS:0;jenkins-hbase4:37265] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:26,724 DEBUG [RS:0;jenkins-hbase4:37265] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:26,724 DEBUG [RS:0;jenkins-hbase4:37265] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:26,724 DEBUG [RS:0;jenkins-hbase4:37265] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:26,726 INFO [RS:0;jenkins-hbase4:37265] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:26,727 INFO [RS:0;jenkins-hbase4:37265] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:26,727 INFO [RS:0;jenkins-hbase4:37265] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:26,727 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=100ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-18 04:54:26,738 INFO [RS:0;jenkins-hbase4:37265] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 04:54:26,738 INFO [RS:0;jenkins-hbase4:37265] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37265,1689656066415-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:26,750 INFO [RS:0;jenkins-hbase4:37265] regionserver.Replication(203): jenkins-hbase4.apache.org,37265,1689656066415 started 2023-07-18 04:54:26,750 INFO [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,37265,1689656066415, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37265, sessionid=0x10176c7bf3c001d 2023-07-18 04:54:26,750 DEBUG [RS:0;jenkins-hbase4:37265] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 04:54:26,750 DEBUG [RS:0;jenkins-hbase4:37265] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:26,750 DEBUG [RS:0;jenkins-hbase4:37265] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37265,1689656066415' 2023-07-18 04:54:26,750 DEBUG [RS:0;jenkins-hbase4:37265] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-18 04:54:26,751 DEBUG [RS:0;jenkins-hbase4:37265] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-18 04:54:26,751 DEBUG [RS:0;jenkins-hbase4:37265] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 04:54:26,751 DEBUG [RS:0;jenkins-hbase4:37265] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 04:54:26,751 DEBUG [RS:0;jenkins-hbase4:37265] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:26,751 DEBUG [RS:0;jenkins-hbase4:37265] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37265,1689656066415' 2023-07-18 04:54:26,751 DEBUG [RS:0;jenkins-hbase4:37265] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-18 04:54:26,752 DEBUG [RS:0;jenkins-hbase4:37265] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-18 04:54:26,752 DEBUG [RS:0;jenkins-hbase4:37265] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 04:54:26,752 INFO [RS:0;jenkins-hbase4:37265] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 04:54:26,752 INFO [RS:0;jenkins-hbase4:37265] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 04:54:26,804 DEBUG [jenkins-hbase4:33929] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-18 04:54:26,805 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37265,1689656066415, state=OPENING 2023-07-18 04:54:26,806 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-18 04:54:26,806 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-18 04:54:26,806 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37265,1689656066415}] 2023-07-18 04:54:26,852 INFO [RS:0;jenkins-hbase4:37265] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:26,854 INFO [RS:0;jenkins-hbase4:37265] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37265%2C1689656066415.jenkins-hbase4.apache.org%2C37265%2C1689656066415.regiongroup-0, suffix=, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs, maxLogs=10 2023-07-18 04:54:26,865 INFO [RS:0;jenkins-hbase4:37265] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415/jenkins-hbase4.apache.org%2C37265%2C1689656066415.jenkins-hbase4.apache.org%2C37265%2C1689656066415.regiongroup-0.1689656066855 2023-07-18 04:54:26,866 DEBUG [RS:0;jenkins-hbase4:37265] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38949,DS-7e9427a5-d2d9-4ece-8da8-f6fb77b8864e,DISK], DatanodeInfoWithStorage[127.0.0.1:37603,DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347,DISK], DatanodeInfoWithStorage[127.0.0.1:40889,DS-cb62bdfa-114c-4e1c-89f3-b40b9c3afb83,DISK]] 2023-07-18 04:54:26,961 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:26,961 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 04:54:26,964 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38552, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 04:54:26,969 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-18 04:54:26,969 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:26,969 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:26,969 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:26,971 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37265%2C1689656066415.meta, suffix=.meta, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs, maxLogs=10 2023-07-18 04:54:26,979 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415/jenkins-hbase4.apache.org%2C37265%2C1689656066415.meta.1689656066971.meta 2023-07-18 04:54:26,979 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37603,DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347,DISK], DatanodeInfoWithStorage[127.0.0.1:44317,DS-d0c3f250-f9ef-4d3a-8da0-904990569608,DISK], DatanodeInfoWithStorage[127.0.0.1:38949,DS-7e9427a5-d2d9-4ece-8da8-f6fb77b8864e,DISK]] 2023-07-18 04:54:26,979 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:26,980 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-18 04:54:26,980 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-18 04:54:26,981 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-18 04:54:26,981 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-18 04:54:26,981 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:26,981 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-18 04:54:26,981 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-18 04:54:26,983 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-18 04:54:26,984 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info 2023-07-18 04:54:26,984 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info 2023-07-18 04:54:26,984 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-18 04:54:26,992 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info/d057a108d301417792b3fe3bc3fbdfc0 2023-07-18 04:54:26,992 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:26,992 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-18 04:54:26,993 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/rep_barrier 2023-07-18 04:54:26,993 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/rep_barrier 2023-07-18 04:54:26,994 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-18 04:54:26,994 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:26,994 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-18 04:54:26,996 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/table 2023-07-18 04:54:26,996 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/table 2023-07-18 04:54:26,996 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-18 04:54:27,003 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/table/bd05a84742aa4c3d88ab2415d20b934f 2023-07-18 04:54:27,003 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:27,004 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740 2023-07-18 04:54:27,006 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740 2023-07-18 04:54:27,008 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-18 04:54:27,010 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-18 04:54:27,011 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69282260, jitterRate=0.03238612413406372}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-18 04:54:27,011 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-18 04:54:27,014 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=20, masterSystemTime=1689656066961 2023-07-18 04:54:27,018 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-18 04:54:27,019 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-18 04:54:27,020 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37265,1689656066415, state=OPEN 2023-07-18 04:54:27,021 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-18 04:54:27,022 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-18 04:54:27,026 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-18 04:54:27,026 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37265,1689656066415 in 216 msec 2023-07-18 04:54:27,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=17 2023-07-18 04:54:27,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 378 msec 2023-07-18 04:54:28,230 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1603ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-18 04:54:29,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3106ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3006ms 2023-07-18 04:54:31,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4508ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-18 04:54:31,135 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-18 04:54:31,139 DEBUG [hconnection-0x4752e652-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:31,142 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48160, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:31,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=383bb30520997d1bb825dc5e5f31999e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42145,1689656058011, regionLocation=jenkins-hbase4.apache.org,42145,1689656058011, openSeqNum=2 2023-07-18 04:54:31,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=a2e71ceab1ce1a583f43e863ff7cd0a0, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42145,1689656058011, regionLocation=jenkins-hbase4.apache.org,42145,1689656058011, openSeqNum=2 2023-07-18 04:54:31,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=1b9326fa16ae8497bb1df12a88dfd577, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42145,1689656058011, regionLocation=jenkins-hbase4.apache.org,42145,1689656058011, openSeqNum=2 2023-07-18 04:54:31,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-18 04:54:31,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689656131152 2023-07-18 04:54:31,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689656191152 2023-07-18 04:54:31,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 16 msec 2023-07-18 04:54:31,175 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33929,1689656066387-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:31,175 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33929,1689656066387-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:31,175 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33929,1689656066387-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:31,175 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33929, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:31,175 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:31,175 INFO [PEWorker-1] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,36167,1689656057977 had 0 regions 2023-07-18 04:54:31,175 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,43893,1689656057910 had 0 regions 2023-07-18 04:54:31,175 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,42145,1689656058011 had 4 regions 2023-07-18 04:54:31,175 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,43065,1689656057946 had 0 regions 2023-07-18 04:54:31,176 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. is NOT online; state={383bb30520997d1bb825dc5e5f31999e state=OPEN, ts=1689656071151, server=jenkins-hbase4.apache.org,42145,1689656058011}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-18 04:54:31,177 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,36167,1689656057977, splitWal=true, meta=false, isMeta: false 2023-07-18 04:54:31,177 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43893,1689656057910, splitWal=true, meta=false, isMeta: false 2023-07-18 04:54:31,178 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=17, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42145,1689656058011, splitWal=true, meta=true, isMeta: false 2023-07-18 04:54:31,178 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43065,1689656057946, splitWal=true, meta=false, isMeta: false 2023-07-18 04:54:31,179 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,36167,1689656057977-splitting 2023-07-18 04:54:31,182 DEBUG [PEWorker-2] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43893,1689656057910-splitting 2023-07-18 04:54:31,183 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,36167,1689656057977-splitting dir is empty, no logs to split. 2023-07-18 04:54:31,183 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,36167,1689656057977 WAL count=0, meta=false 2023-07-18 04:54:31,184 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43893,1689656057910-splitting dir is empty, no logs to split. 2023-07-18 04:54:31,184 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,43893,1689656057910 WAL count=0, meta=false 2023-07-18 04:54:31,184 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,42145,1689656058011/hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e., unknown_server=jenkins-hbase4.apache.org,42145,1689656058011/t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0., unknown_server=jenkins-hbase4.apache.org,42145,1689656058011/t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:31,185 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011-splitting dir is empty, no logs to split. 2023-07-18 04:54:31,185 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,42145,1689656058011 WAL count=0, meta=false 2023-07-18 04:54:31,185 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43065,1689656057946-splitting 2023-07-18 04:54:31,186 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43065,1689656057946-splitting dir is empty, no logs to split. 2023-07-18 04:54:31,186 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,43065,1689656057946 WAL count=0, meta=false 2023-07-18 04:54:31,187 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,36167,1689656057977-splitting dir is empty, no logs to split. 2023-07-18 04:54:31,187 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,36167,1689656057977 WAL count=0, meta=false 2023-07-18 04:54:31,187 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,36167,1689656057977 WAL splitting is done? wals=0, meta=false 2023-07-18 04:54:31,188 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43893,1689656057910-splitting dir is empty, no logs to split. 2023-07-18 04:54:31,188 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,43893,1689656057910 WAL count=0, meta=false 2023-07-18 04:54:31,188 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,43893,1689656057910 WAL splitting is done? wals=0, meta=false 2023-07-18 04:54:31,190 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,42145,1689656058011-splitting dir is empty, no logs to split. 2023-07-18 04:54:31,190 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,42145,1689656058011 WAL count=0, meta=false 2023-07-18 04:54:31,190 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,42145,1689656058011 WAL splitting is done? wals=0, meta=false 2023-07-18 04:54:31,192 INFO [PEWorker-2] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,43893,1689656057910 failed, ignore...File hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43893,1689656057910-splitting does not exist. 2023-07-18 04:54:31,192 INFO [PEWorker-1] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,36167,1689656057977 failed, ignore...File hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,36167,1689656057977-splitting does not exist. 2023-07-18 04:54:31,196 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43065,1689656057946-splitting dir is empty, no logs to split. 2023-07-18 04:54:31,196 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,43065,1689656057946 WAL count=0, meta=false 2023-07-18 04:54:31,196 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,43065,1689656057946 WAL splitting is done? wals=0, meta=false 2023-07-18 04:54:31,197 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=383bb30520997d1bb825dc5e5f31999e, ASSIGN}, {pid=22, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=1b9326fa16ae8497bb1df12a88dfd577, ASSIGN}, {pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=a2e71ceab1ce1a583f43e863ff7cd0a0, ASSIGN}] 2023-07-18 04:54:31,198 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,36167,1689656057977 after splitting done 2023-07-18 04:54:31,198 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,43893,1689656057910 after splitting done 2023-07-18 04:54:31,198 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,36167,1689656057977 from processing; numProcessing=3 2023-07-18 04:54:31,198 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,43893,1689656057910 from processing; numProcessing=2 2023-07-18 04:54:31,198 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=383bb30520997d1bb825dc5e5f31999e, ASSIGN 2023-07-18 04:54:31,199 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,43065,1689656057946 failed, ignore...File hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,43065,1689656057946-splitting does not exist. 2023-07-18 04:54:31,199 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=1b9326fa16ae8497bb1df12a88dfd577, ASSIGN 2023-07-18 04:54:31,200 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=21, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=383bb30520997d1bb825dc5e5f31999e, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 04:54:31,200 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=1b9326fa16ae8497bb1df12a88dfd577, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 04:54:31,201 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=a2e71ceab1ce1a583f43e863ff7cd0a0, ASSIGN 2023-07-18 04:54:31,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,36167,1689656057977, splitWal=true, meta=false in 4.6180 sec 2023-07-18 04:54:31,201 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,43893,1689656057910, splitWal=true, meta=false in 4.6120 sec 2023-07-18 04:54:31,202 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,43065,1689656057946 after splitting done 2023-07-18 04:54:31,202 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,43065,1689656057946 from processing; numProcessing=1 2023-07-18 04:54:31,202 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=a2e71ceab1ce1a583f43e863ff7cd0a0, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 04:54:31,204 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,43065,1689656057946, splitWal=true, meta=false in 4.6350 sec 2023-07-18 04:54:31,350 DEBUG [jenkins-hbase4:33929] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-18 04:54:31,353 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=383bb30520997d1bb825dc5e5f31999e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:31,353 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=a2e71ceab1ce1a583f43e863ff7cd0a0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:31,353 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=1b9326fa16ae8497bb1df12a88dfd577, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:31,353 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689656071352"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656071352"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656071352"}]},"ts":"1689656071352"} 2023-07-18 04:54:31,353 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656071353"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656071353"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656071353"}]},"ts":"1689656071353"} 2023-07-18 04:54:31,353 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656071353"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656071353"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656071353"}]},"ts":"1689656071353"} 2023-07-18 04:54:31,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=21, state=RUNNABLE; OpenRegionProcedure 383bb30520997d1bb825dc5e5f31999e, server=jenkins-hbase4.apache.org,37265,1689656066415}] 2023-07-18 04:54:31,358 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure 1b9326fa16ae8497bb1df12a88dfd577, server=jenkins-hbase4.apache.org,37265,1689656066415}] 2023-07-18 04:54:31,359 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=23, state=RUNNABLE; OpenRegionProcedure a2e71ceab1ce1a583f43e863ff7cd0a0, server=jenkins-hbase4.apache.org,37265,1689656066415}] 2023-07-18 04:54:31,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:31,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:31,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37265%2C1689656066415.jenkins-hbase4.apache.org%2C37265%2C1689656066415.regiongroup-1, suffix=, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs, maxLogs=10 2023-07-18 04:54:31,523 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:31,523 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 383bb30520997d1bb825dc5e5f31999e, NAME => 'hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:31,523 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:31,523 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:31,523 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:31,523 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:31,524 INFO [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:31,526 DEBUG [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/info 2023-07-18 04:54:31,526 DEBUG [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/info 2023-07-18 04:54:31,526 INFO [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 383bb30520997d1bb825dc5e5f31999e columnFamilyName info 2023-07-18 04:54:31,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415/jenkins-hbase4.apache.org%2C37265%2C1689656066415.jenkins-hbase4.apache.org%2C37265%2C1689656066415.regiongroup-1.1689656071517 2023-07-18 04:54:31,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37603,DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347,DISK], DatanodeInfoWithStorage[127.0.0.1:44317,DS-d0c3f250-f9ef-4d3a-8da0-904990569608,DISK], DatanodeInfoWithStorage[127.0.0.1:38949,DS-7e9427a5-d2d9-4ece-8da8-f6fb77b8864e,DISK]] 2023-07-18 04:54:31,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a2e71ceab1ce1a583f43e863ff7cd0a0, NAME => 't1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:31,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:31,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:31,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:31,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:31,530 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:31,531 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1 2023-07-18 04:54:31,531 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1 2023-07-18 04:54:31,532 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2e71ceab1ce1a583f43e863ff7cd0a0 columnFamilyName cf1 2023-07-18 04:54:31,535 DEBUG [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/info/fad48ef599d54b4b956ee92103e35a9a 2023-07-18 04:54:31,535 INFO [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] regionserver.HStore(310): Store=383bb30520997d1bb825dc5e5f31999e/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:31,536 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:31,537 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:31,540 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:31,541 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1/cc4f63e41e55476a976035dc43508488 2023-07-18 04:54:31,541 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(310): Store=a2e71ceab1ce1a583f43e863ff7cd0a0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:31,541 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:31,541 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 383bb30520997d1bb825dc5e5f31999e; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63127773, jitterRate=-0.05932287871837616}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 04:54:31,541 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 383bb30520997d1bb825dc5e5f31999e: 2023-07-18 04:54:31,544 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/norep 2023-07-18 04:54:31,544 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/norep 2023-07-18 04:54:31,544 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2e71ceab1ce1a583f43e863ff7cd0a0 columnFamilyName norep 2023-07-18 04:54:31,544 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e., pid=24, masterSystemTime=1689656071509 2023-07-18 04:54:31,545 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(310): Store=a2e71ceab1ce1a583f43e863ff7cd0a0/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:31,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:31,548 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:31,548 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:31,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:31,549 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=383bb30520997d1bb825dc5e5f31999e, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:31,549 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689656071549"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656071549"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656071549"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656071549"}]},"ts":"1689656071549"} 2023-07-18 04:54:31,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 04:54:31,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:31,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a2e71ceab1ce1a583f43e863ff7cd0a0; next sequenceid=11; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59309809, jitterRate=-0.11621497571468353}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 04:54:31,553 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a2e71ceab1ce1a583f43e863ff7cd0a0: 2023-07-18 04:54:31,555 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=21 2023-07-18 04:54:31,555 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=21, state=SUCCESS; OpenRegionProcedure 383bb30520997d1bb825dc5e5f31999e, server=jenkins-hbase4.apache.org,37265,1689656066415 in 195 msec 2023-07-18 04:54:31,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0., pid=26, masterSystemTime=1689656071509 2023-07-18 04:54:31,557 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=383bb30520997d1bb825dc5e5f31999e, ASSIGN in 358 msec 2023-07-18 04:54:31,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:31,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:31,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:31,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1b9326fa16ae8497bb1df12a88dfd577, NAME => 't2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:31,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:31,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:31,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:31,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:31,558 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=a2e71ceab1ce1a583f43e863ff7cd0a0, regionState=OPEN, openSeqNum=11, regionLocation=jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:31,558 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656071558"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656071558"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656071558"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656071558"}]},"ts":"1689656071558"} 2023-07-18 04:54:31,559 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:31,560 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1 2023-07-18 04:54:31,560 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1 2023-07-18 04:54:31,560 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b9326fa16ae8497bb1df12a88dfd577 columnFamilyName cf1 2023-07-18 04:54:31,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=23 2023-07-18 04:54:31,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=23, state=SUCCESS; OpenRegionProcedure a2e71ceab1ce1a583f43e863ff7cd0a0, server=jenkins-hbase4.apache.org,37265,1689656066415 in 201 msec 2023-07-18 04:54:31,564 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=a2e71ceab1ce1a583f43e863ff7cd0a0, ASSIGN in 366 msec 2023-07-18 04:54:31,568 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1/bbbbe1a481884a64ac20a4d86fe979a7 2023-07-18 04:54:31,568 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(310): Store=1b9326fa16ae8497bb1df12a88dfd577/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:31,568 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:31,569 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/norep 2023-07-18 04:54:31,569 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/norep 2023-07-18 04:54:31,570 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b9326fa16ae8497bb1df12a88dfd577 columnFamilyName norep 2023-07-18 04:54:31,570 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(310): Store=1b9326fa16ae8497bb1df12a88dfd577/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:31,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:31,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:31,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 04:54:31,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:31,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1b9326fa16ae8497bb1df12a88dfd577; next sequenceid=12; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63042839, jitterRate=-0.060588493943214417}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 04:54:31,577 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1b9326fa16ae8497bb1df12a88dfd577: 2023-07-18 04:54:31,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577., pid=25, masterSystemTime=1689656071509 2023-07-18 04:54:31,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:31,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:31,581 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=1b9326fa16ae8497bb1df12a88dfd577, regionState=OPEN, openSeqNum=12, regionLocation=jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:31,581 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656071581"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656071581"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656071581"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656071581"}]},"ts":"1689656071581"} 2023-07-18 04:54:31,585 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-07-18 04:54:31,585 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure 1b9326fa16ae8497bb1df12a88dfd577, server=jenkins-hbase4.apache.org,37265,1689656066415 in 225 msec 2023-07-18 04:54:31,588 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=17 2023-07-18 04:54:31,588 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,42145,1689656058011 after splitting done 2023-07-18 04:54:31,588 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=1b9326fa16ae8497bb1df12a88dfd577, ASSIGN in 388 msec 2023-07-18 04:54:31,588 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,42145,1689656058011 from processing; numProcessing=0 2023-07-18 04:54:31,590 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,42145,1689656058011, splitWal=true, meta=true in 5.0040 sec 2023-07-18 04:54:31,683 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 04:54:31,748 INFO [jenkins-hbase4:37265Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 04:54:32,177 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-18 04:54:32,192 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-18 04:54:32,195 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-18 04:54:32,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.753sec 2023-07-18 04:54:32,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-18 04:54:32,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-18 04:54:32,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-18 04:54:32,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33929,1689656066387-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-18 04:54:32,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33929,1689656066387-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-18 04:54:32,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-18 04:54:32,241 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(139): Connect 0x671b6018 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:32,247 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@8977322, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:32,248 DEBUG [hconnection-0x3bfa0f9f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:32,251 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48162, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:32,254 INFO [Listener at localhost/37011] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-18 04:54:32,254 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x671b6018 to 127.0.0.1:63236 2023-07-18 04:54:32,254 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:32,255 INFO [Listener at localhost/37011] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:33929 after: jenkins-hbase4.apache.org:33929 2023-07-18 04:54:32,256 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(139): Connect 0x4474cedf to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:32,260 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f99a739, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:32,699 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-18 04:54:32,700 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-18 04:54:32,701 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-18 04:54:32,701 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-18 04:54:32,762 DEBUG [hconnection-0x14b12e67-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:32,767 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48170, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:32,794 INFO [Listener at localhost/37011] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689656072793 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:32,799 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): syncupReplication16896560727930x0, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:32,799 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689656072793-0x10176c7bf3c0022 connected Start Replication Server start 2023-07-18 04:54:32,803 INFO [Listener at localhost/37011] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:32,803 INFO [Listener at localhost/37011] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:32,815 INFO [Listener at localhost/37011] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-18 04:54:32,816 DEBUG [Listener at localhost/37011] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 04:54:32,829 INFO [Listener at localhost/37011] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-18 04:54:32,830 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,42175,1689656054096, jenkins-hbase4.apache.org,45857,1689656054144] other RSs: [] 2023-07-18 04:54:32,850 DEBUG [Listener at localhost/37011.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x6102bfa2 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:32,856 DEBUG [Listener at localhost/37011.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@155c0287, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:32,856 INFO [Listener at localhost/37011.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:32,859 DEBUG [Listener at localhost/37011.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:32,860 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10176c7bf3c0024 connected 2023-07-18 04:54:32,860 INFO [Listener at localhost/37011.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=4b2ea01c-df7d-4e18-8185-4db3e50f17db to cluster=b492e8cd-f4f9-4591-97c6-7f2edd4114a8 2023-07-18 04:54:36,417 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,42175,1689656054096/1's WALs to 1689656072803.syncuptool.replication.org,1234,1 2023-07-18 04:54:36,419 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(403): Removed empty jenkins-hbase4.apache.org,42175,1689656054096/1 2023-07-18 04:54:36,748 INFO [jenkins-hbase4:37265Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 04:54:37,027 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 04:54:39,694 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,45857,1689656054144/1's WALs to 1689656072803.syncuptool.replication.org,1234,1 2023-07-18 04:54:39,697 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0.1689656055500 with data PBUF\x08\xDFU 2023-07-18 04:54:39,698 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1.1689656056513 with data PBUF\x08\xFD\xAD\x01 2023-07-18 04:54:39,699 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,45857,1689656054144/1's WALs to 1689656072803.syncuptool.replication.org,1234,1 2023-07-18 04:54:41,701 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,45857,1689656054144] 2023-07-18 04:54:41,702 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,45857,1689656054144] 2023-07-18 04:54:41,717 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,45857,1689656054144] 2023-07-18 04:54:41,717 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,45857,1689656054144, ReplicationSource: 1, currentBandwidth=0 2023-07-18 04:54:41,717 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:41,737 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144] zookeeper.ReadOnlyZKClient(139): Connect 0x65cff136 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:41,743 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@24878104, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:41,743 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:41,748 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:41,748 INFO [jenkins-hbase4:37265Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 04:54:41,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10176c7bf3c0026 connected 2023-07-18 04:54:41,749 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,45857,1689656054144 (queues=2) is replicating from cluster=4b2ea01c-df7d-4e18-8185-4db3e50f17db to cluster=b492e8cd-f4f9-4591-97c6-7f2edd4114a8 2023-07-18 04:54:41,750 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1 2023-07-18 04:54:41,754 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,45857,1689656054144, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 04:54:41,754 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0 2023-07-18 04:54:41,754 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1 2023-07-18 04:54:41,756 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,45857,1689656054144, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 04:54:41,757 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0 2023-07-18 04:54:41,763 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0.1689656055500 2023-07-18 04:54:41,763 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1.1689656056513 2023-07-18 04:54:41,763 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-18 04:54:41,764 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-18 04:54:41,764 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-18 04:54:41,764 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-18 04:54:41,765 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,45857,1689656054144 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/db11400df50c3fb2fe7bab3791162e0a/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0.1689656055500, lastWalPosition=15173, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=true,usedBufferSize=4120] 2023-07-18 04:54:41,766 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,45857,1689656054144 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs/jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1.1689656056513, lastWalPosition=30605, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=true,usedBufferSize=8120] 2023-07-18 04:54:41,768 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 04:54:41,774 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35468, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 04:54:41,775 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37265] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 04:54:41,775 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=37265] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 04:54:41,776 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37265] zookeeper.ReadOnlyZKClient(139): Connect 0x6416e408 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:41,783 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37265] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@6ad106fa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:41,788 DEBUG [hconnection-0x6eee8f78-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:41,791 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35484, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=ClientService 2023-07-18 04:54:41,800 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37265] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 04:54:41,801 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1-jenkins-hbase4.apache.org,45857,1689656054144] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,45857,1689656054144] 2023-07-18 04:54:41,803 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0.1689656055500] 2023-07-18 04:54:41,805 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=37265] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 04:54:41,805 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1-jenkins-hbase4.apache.org,45857,1689656054144] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,45857,1689656054144] 2023-07-18 04:54:41,812 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,45857,1689656054144 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-18 04:54:41,812 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:41,813 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1.1689656056513] 2023-07-18 04:54:41,814 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,45857,1689656054144 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-18 04:54:41,814 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:41,913 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:41,917 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689656072793-0x10176c7bf3c0022, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689656072803.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:41,918 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-0,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,45857,1689656054144 with the following stats: Total replicated edits: 2, current progress: 2023-07-18 04:54:42,018 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,45857,1689656054144 2023-07-18 04:54:42,019 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1-jenkins-hbase4.apache.org,45857,1689656054144] zookeeper.ZKUtil(243): syncupReplication1689656072793-0x10176c7bf3c0022, quorum=127.0.0.1:63236, baseZNode=/1 Unable to list children of znode /1/replication/rs/1689656072803.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,45857,1689656054144 because node does not exist (not an error) 2023-07-18 04:54:42,019 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144.replicationSource.shipperjenkins-hbase4.apache.org%2C45857%2C1689656054144.jenkins-hbase4.apache.org%2C45857%2C1689656054144.regiongroup-1,1-jenkins-hbase4.apache.org,45857,1689656054144] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,45857,1689656054144 with the following stats: Total replicated edits: 2, current progress: 2023-07-18 04:54:42,832 INFO [Listener at localhost/37011] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-18 04:54:42,936 DEBUG [Listener at localhost/37011.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10176c7bf3c0024, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:42,936 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6102bfa2 to 127.0.0.1:63236 2023-07-18 04:54:42,937 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:43,139 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689656072793-0x10176c7bf3c0022, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:43,139 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689656072793-0x10176c7bf3c0022, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:43,151 INFO [Listener at localhost/37011] replication.TestReplicationSyncUpTool(203): SyncUpAfterDelete succeeded at retry = 0 2023-07-18 04:54:43,151 DEBUG [Listener at localhost/37011] replication.TestReplicationSyncUpTool(215): mimicSyncUpAfterPut 2023-07-18 04:54:43,153 INFO [Listener at localhost/37011] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-18 04:54:43,153 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:43,153 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:43,153 INFO [Listener at localhost/37011] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 04:54:43,154 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:43,154 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 04:54:43,154 INFO [Listener at localhost/37011] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 04:54:43,157 INFO [Listener at localhost/37011] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36243 2023-07-18 04:54:43,157 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:43,159 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:43,160 INFO [Listener at localhost/37011] zookeeper.RecoverableZooKeeper(93): Process identifier=master:36243 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:43,163 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:362430x0, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:43,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:36243-0x10176c7bf3c0028 connected 2023-07-18 04:54:43,167 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-18 04:54:43,169 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 04:54:43,169 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-18 04:54:43,171 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36243 2023-07-18 04:54:43,171 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36243 2023-07-18 04:54:43,171 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36243 2023-07-18 04:54:43,171 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36243 2023-07-18 04:54:43,172 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36243 2023-07-18 04:54:43,172 INFO [Listener at localhost/37011] master.HMaster(444): hbase.rootdir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c, hbase.cluster.distributed=false 2023-07-18 04:54:43,173 DEBUG [pool-550-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: INIT 2023-07-18 04:54:43,191 INFO [Listener at localhost/37011] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-18 04:54:43,191 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:43,191 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:43,191 INFO [Listener at localhost/37011] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 04:54:43,191 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:43,191 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 04:54:43,192 INFO [Listener at localhost/37011] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 04:54:43,194 INFO [Listener at localhost/37011] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45615 2023-07-18 04:54:43,195 INFO [Listener at localhost/37011] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 04:54:43,196 DEBUG [Listener at localhost/37011] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 04:54:43,196 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:43,197 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:43,198 INFO [Listener at localhost/37011] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45615 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:43,200 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:456150x0, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:43,202 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:456150x0, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-18 04:54:43,203 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45615-0x10176c7bf3c0029 connected 2023-07-18 04:54:43,203 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:45615-0x10176c7bf3c0029, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 04:54:43,203 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:45615-0x10176c7bf3c0029, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-18 04:54:43,207 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45615 2023-07-18 04:54:43,207 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45615 2023-07-18 04:54:43,207 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45615 2023-07-18 04:54:43,207 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45615 2023-07-18 04:54:43,210 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45615 2023-07-18 04:54:43,214 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,36243,1689656083152 2023-07-18 04:54:43,215 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-18 04:54:43,216 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,36243,1689656083152 2023-07-18 04:54:43,218 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:45615-0x10176c7bf3c0029, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-18 04:54:43,218 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-18 04:54:43,219 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:43,219 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-18 04:54:43,221 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,36243,1689656083152 from backup master directory 2023-07-18 04:54:43,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-18 04:54:43,222 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,36243,1689656083152 2023-07-18 04:54:43,222 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 04:54:43,222 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-18 04:54:43,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,36243,1689656083152 2023-07-18 04:54:43,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:43,265 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x3aa5c70f to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:43,269 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@36e1c165, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:43,269 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 04:54:43,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-18 04:54:43,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:43,271 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:43,277 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/WALs/jenkins-hbase4.apache.org,44113,1689656052898 to hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/WALs/jenkins-hbase4.apache.org,44113,1689656052898-dead as it is dead 2023-07-18 04:54:43,277 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/WALs/jenkins-hbase4.apache.org,44113,1689656052898-dead/jenkins-hbase4.apache.org%2C44113%2C1689656052898.jenkins-hbase4.apache.org%2C44113%2C1689656052898.regiongroup-0.1689656054594 2023-07-18 04:54:43,278 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/WALs/jenkins-hbase4.apache.org,44113,1689656052898-dead/jenkins-hbase4.apache.org%2C44113%2C1689656052898.jenkins-hbase4.apache.org%2C44113%2C1689656052898.regiongroup-0.1689656054594 after 1ms 2023-07-18 04:54:43,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/WALs/jenkins-hbase4.apache.org,44113,1689656052898-dead/jenkins-hbase4.apache.org%2C44113%2C1689656052898.jenkins-hbase4.apache.org%2C44113%2C1689656052898.regiongroup-0.1689656054594 to hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C44113%2C1689656052898.jenkins-hbase4.apache.org%2C44113%2C1689656052898.regiongroup-0.1689656054594 2023-07-18 04:54:43,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/WALs/jenkins-hbase4.apache.org,44113,1689656052898-dead 2023-07-18 04:54:43,279 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/WALs/jenkins-hbase4.apache.org,36243,1689656083152 2023-07-18 04:54:43,280 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:43,282 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36243%2C1689656083152.jenkins-hbase4.apache.org%2C36243%2C1689656083152.regiongroup-0, suffix=, logDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/WALs/jenkins-hbase4.apache.org,36243,1689656083152, archiveDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/oldWALs, maxLogs=10 2023-07-18 04:54:43,296 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/WALs/jenkins-hbase4.apache.org,36243,1689656083152/jenkins-hbase4.apache.org%2C36243%2C1689656083152.jenkins-hbase4.apache.org%2C36243%2C1689656083152.regiongroup-0.1689656083283 2023-07-18 04:54:43,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36951,DS-63fc3a07-0ff5-4e01-8767-68d629b125e2,DISK], DatanodeInfoWithStorage[127.0.0.1:34717,DS-15daf992-9329-4b25-9667-a096fb2deda0,DISK]] 2023-07-18 04:54:43,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:43,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:43,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:43,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:43,298 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:43,299 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-18 04:54:43,300 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-18 04:54:43,307 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3e1e56add1154f2eb5422b15934f49bc 2023-07-18 04:54:43,307 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:43,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-18 04:54:43,308 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C44113%2C1689656052898.jenkins-hbase4.apache.org%2C44113%2C1689656052898.regiongroup-0.1689656054594 2023-07-18 04:54:43,314 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 155, firstSequenceIdInLog=3, maxSequenceIdInLog=138, path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C44113%2C1689656052898.jenkins-hbase4.apache.org%2C44113%2C1689656052898.regiongroup-0.1689656054594 2023-07-18 04:54:43,317 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C44113%2C1689656052898.jenkins-hbase4.apache.org%2C44113%2C1689656052898.regiongroup-0.1689656054594 2023-07-18 04:54:43,321 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:43,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/138.seqid, newMaxSeqId=138, maxSeqId=1 2023-07-18 04:54:43,325 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=139; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71175943, jitterRate=0.06060419976711273}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 04:54:43,325 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 04:54:43,325 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-18 04:54:43,327 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-18 04:54:43,327 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-18 04:54:43,327 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-18 04:54:43,327 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-18 04:54:43,332 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-18 04:54:43,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-18 04:54:43,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-18 04:54:43,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-18 04:54:43,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-18 04:54:43,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-18 04:54:43,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-18 04:54:43,334 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-18 04:54:43,334 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-18 04:54:43,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-18 04:54:43,340 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42175,1689656054096, table=hbase:meta, region=1588230740 2023-07-18 04:54:43,341 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 2 possibly 'live' servers, and 0 'splitting'. 2023-07-18 04:54:43,343 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,42175,1689656054096 already deleted, retry=false 2023-07-18 04:54:43,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,42175,1689656054096 on jenkins-hbase4.apache.org,36243,1689656083152 2023-07-18 04:54:43,346 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,42175,1689656054096, splitWal=true, meta=true 2023-07-18 04:54:43,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,42175,1689656054096 (carryingMeta=true) jenkins-hbase4.apache.org,42175,1689656054096/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@7c87a51a[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-18 04:54:43,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,45857,1689656054144 already deleted, retry=false 2023-07-18 04:54:43,348 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,45857,1689656054144 on jenkins-hbase4.apache.org,36243,1689656083152 2023-07-18 04:54:43,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,45857,1689656054144, splitWal=true, meta=false 2023-07-18 04:54:43,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=19 for jenkins-hbase4.apache.org,45857,1689656054144 (carryingMeta=false) jenkins-hbase4.apache.org,45857,1689656054144/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@1bab34d5[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-18 04:54:43,360 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-18 04:54:43,360 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-18 04:54:43,360 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-18 04:54:43,360 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-18 04:54:43,361 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-18 04:54:43,361 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-18 04:54:43,362 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-18 04:54:43,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-18 04:54:43,365 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:45615-0x10176c7bf3c0029, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-18 04:54:43,365 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-18 04:54:43,366 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:43,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,36243,1689656083152, sessionid=0x10176c7bf3c0028, setting cluster-up flag (Was=false) 2023-07-18 04:54:43,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-18 04:54:43,373 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,36243,1689656083152 2023-07-18 04:54:43,376 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-18 04:54:43,376 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,36243,1689656083152 2023-07-18 04:54:43,377 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/.hbase-snapshot/.tmp 2023-07-18 04:54:43,378 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:43,378 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:43,378 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:43,379 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:43,379 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-18 04:54:43,379 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:43,379 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 04:54:43,379 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:43,388 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689656113388 2023-07-18 04:54:43,388 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-18 04:54:43,390 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-18 04:54:43,390 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-18 04:54:43,390 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-18 04:54:43,390 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-18 04:54:43,390 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-18 04:54:43,390 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:43,392 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45857,1689656054144; numProcessing=1 2023-07-18 04:54:43,393 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42175,1689656054096; numProcessing=2 2023-07-18 04:54:43,393 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-18 04:54:43,393 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42175,1689656054096, splitWal=true, meta=true 2023-07-18 04:54:43,394 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-18 04:54:43,393 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=19, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45857,1689656054144, splitWal=true, meta=false 2023-07-18 04:54:43,394 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-18 04:54:43,394 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-18 04:54:43,395 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-18 04:54:43,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656083395,5,FailOnTimeoutGroup] 2023-07-18 04:54:43,395 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42175,1689656054096, splitWal=true, meta=true, isMeta: true 2023-07-18 04:54:43,398 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656083395,5,FailOnTimeoutGroup] 2023-07-18 04:54:43,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:43,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-18 04:54:43,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:43,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:43,400 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689656083400, completionTime=-1 2023-07-18 04:54:43,400 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-18 04:54:43,400 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-18 04:54:43,401 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096-splitting 2023-07-18 04:54:43,402 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096-splitting dir is empty, no logs to split. 2023-07-18 04:54:43,402 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,42175,1689656054096 WAL count=0, meta=true 2023-07-18 04:54:43,404 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096-splitting dir is empty, no logs to split. 2023-07-18 04:54:43,404 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,42175,1689656054096 WAL count=0, meta=true 2023-07-18 04:54:43,404 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,42175,1689656054096 WAL splitting is done? wals=0, meta=true 2023-07-18 04:54:43,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-18 04:54:43,407 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-18 04:54:43,409 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 04:54:43,413 INFO [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(951): ClusterId : 4b2ea01c-df7d-4e18-8185-4db3e50f17db 2023-07-18 04:54:43,413 DEBUG [RS:0;jenkins-hbase4:45615] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 04:54:43,415 DEBUG [RS:0;jenkins-hbase4:45615] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 04:54:43,415 DEBUG [RS:0;jenkins-hbase4:45615] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 04:54:43,417 DEBUG [RS:0;jenkins-hbase4:45615] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 04:54:43,420 DEBUG [RS:0;jenkins-hbase4:45615] zookeeper.ReadOnlyZKClient(139): Connect 0x77d0ac0d to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:43,427 DEBUG [RS:0;jenkins-hbase4:45615] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@79175c87, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:43,427 DEBUG [RS:0;jenkins-hbase4:45615] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7d28c77d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 04:54:43,435 DEBUG [RS:0;jenkins-hbase4:45615] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:45615 2023-07-18 04:54:43,435 INFO [RS:0;jenkins-hbase4:45615] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 04:54:43,435 INFO [RS:0;jenkins-hbase4:45615] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 04:54:43,435 DEBUG [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 04:54:43,436 INFO [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,36243,1689656083152 with isa=jenkins-hbase4.apache.org/172.31.14.131:45615, startcode=1689656083190 2023-07-18 04:54:43,436 DEBUG [RS:0;jenkins-hbase4:45615] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 04:54:43,443 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52749, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 04:54:43,444 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36243] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:43,445 DEBUG [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c 2023-07-18 04:54:43,445 DEBUG [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:32937 2023-07-18 04:54:43,445 DEBUG [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 04:54:43,446 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 04:54:43,447 DEBUG [RS:0;jenkins-hbase4:45615] zookeeper.ZKUtil(162): regionserver:45615-0x10176c7bf3c0029, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:43,447 WARN [RS:0;jenkins-hbase4:45615] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 04:54:43,447 INFO [RS:0;jenkins-hbase4:45615] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:43,448 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45615,1689656083190] 2023-07-18 04:54:43,448 INFO [RS:0;jenkins-hbase4:45615] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:43,448 DEBUG [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(1948): logDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:43,450 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-18 04:54:43,466 INFO [RS:0;jenkins-hbase4:45615] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-18 04:54:43,467 DEBUG [RS:0;jenkins-hbase4:45615] zookeeper.ZKUtil(162): regionserver:45615-0x10176c7bf3c0029, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:43,468 DEBUG [RS:0;jenkins-hbase4:45615] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 04:54:43,468 INFO [RS:0;jenkins-hbase4:45615] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 04:54:43,469 INFO [RS:0;jenkins-hbase4:45615] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 04:54:43,470 INFO [RS:0;jenkins-hbase4:45615] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 04:54:43,470 INFO [RS:0;jenkins-hbase4:45615] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:43,470 INFO [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 04:54:43,472 INFO [RS:0;jenkins-hbase4:45615] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:43,473 DEBUG [RS:0;jenkins-hbase4:45615] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:43,473 DEBUG [RS:0;jenkins-hbase4:45615] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:43,473 DEBUG [RS:0;jenkins-hbase4:45615] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:43,473 DEBUG [RS:0;jenkins-hbase4:45615] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:43,473 DEBUG [RS:0;jenkins-hbase4:45615] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:43,473 DEBUG [RS:0;jenkins-hbase4:45615] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 04:54:43,473 DEBUG [RS:0;jenkins-hbase4:45615] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:43,473 DEBUG [RS:0;jenkins-hbase4:45615] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:43,473 DEBUG [RS:0;jenkins-hbase4:45615] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:43,473 DEBUG [RS:0;jenkins-hbase4:45615] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:43,474 INFO [RS:0;jenkins-hbase4:45615] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:43,475 INFO [RS:0;jenkins-hbase4:45615] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:43,475 INFO [RS:0;jenkins-hbase4:45615] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:43,486 INFO [RS:0;jenkins-hbase4:45615] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 04:54:43,486 INFO [RS:0;jenkins-hbase4:45615] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45615,1689656083190-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:43,496 INFO [RS:0;jenkins-hbase4:45615] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-18 04:54:43,499 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [1689656072803.syncuptool.replication.org,1234,1] other RSs: [jenkins-hbase4.apache.org,45615,1689656083190] 2023-07-18 04:54:43,513 INFO [RS:0;jenkins-hbase4:45615] regionserver.Replication(203): jenkins-hbase4.apache.org,45615,1689656083190 started 2023-07-18 04:54:43,514 INFO [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,45615,1689656083190, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45615, sessionid=0x10176c7bf3c0029 2023-07-18 04:54:43,514 DEBUG [RS:0;jenkins-hbase4:45615] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 04:54:43,514 DEBUG [RS:0;jenkins-hbase4:45615] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:43,514 DEBUG [RS:0;jenkins-hbase4:45615] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45615,1689656083190' 2023-07-18 04:54:43,514 DEBUG [RS:0;jenkins-hbase4:45615] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-18 04:54:43,514 DEBUG [RS:0;jenkins-hbase4:45615] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-18 04:54:43,515 DEBUG [RS:0;jenkins-hbase4:45615] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 04:54:43,515 DEBUG [RS:0;jenkins-hbase4:45615] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 04:54:43,515 DEBUG [RS:0;jenkins-hbase4:45615] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:43,515 DEBUG [RS:0;jenkins-hbase4:45615] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45615,1689656083190' 2023-07-18 04:54:43,515 DEBUG [RS:0;jenkins-hbase4:45615] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-18 04:54:43,515 DEBUG [RS:0;jenkins-hbase4:45615] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-18 04:54:43,516 DEBUG [RS:0;jenkins-hbase4:45615] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 04:54:43,516 INFO [RS:0;jenkins-hbase4:45615] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 04:54:43,516 INFO [RS:0;jenkins-hbase4:45615] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 04:54:43,522 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x0d80e891 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:43,526 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@1a935253, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:43,527 INFO [RS:0;jenkins-hbase4:45615.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:43,529 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:43,530 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10176c7bf3c002d connected 2023-07-18 04:54:43,530 INFO [RS:0;jenkins-hbase4:45615.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=4b2ea01c-df7d-4e18-8185-4db3e50f17db to cluster=b492e8cd-f4f9-4591-97c6-7f2edd4114a8 2023-07-18 04:54:43,559 DEBUG [jenkins-hbase4:36243] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-18 04:54:43,560 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45615,1689656083190, state=OPENING 2023-07-18 04:54:43,563 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-18 04:54:43,563 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-18 04:54:43,563 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45615,1689656083190}] 2023-07-18 04:54:43,616 INFO [RS:0;jenkins-hbase4:45615] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:43,618 INFO [RS:0;jenkins-hbase4:45615] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0, suffix=, logDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190, archiveDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs, maxLogs=10 2023-07-18 04:54:43,634 DEBUG [RS:0;jenkins-hbase4:45615] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0 for peer 1 2023-07-18 04:54:43,634 INFO [RS:0;jenkins-hbase4:45615] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190/jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0.1689656083619 2023-07-18 04:54:43,634 DEBUG [RS:0;jenkins-hbase4:45615] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0 2023-07-18 04:54:43,634 INFO [RS:0;jenkins-hbase4:45615] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 04:54:43,634 DEBUG [RS:0;jenkins-hbase4:45615] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36951,DS-63fc3a07-0ff5-4e01-8767-68d629b125e2,DISK], DatanodeInfoWithStorage[127.0.0.1:34717,DS-15daf992-9329-4b25-9667-a096fb2deda0,DISK]] 2023-07-18 04:54:43,635 INFO [RS:0;jenkins-hbase4:45615.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0 2023-07-18 04:54:43,642 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:43,721 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:43,721 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 04:54:43,723 INFO [RS-EventLoopGroup-14-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58074, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 04:54:43,727 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-18 04:54:43,727 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:43,727 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:43,728 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:43,729 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45615%2C1689656083190.meta, suffix=.meta, logDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190, archiveDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs, maxLogs=10 2023-07-18 04:54:43,740 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190/jenkins-hbase4.apache.org%2C45615%2C1689656083190.meta.1689656083730.meta 2023-07-18 04:54:43,740 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36951,DS-63fc3a07-0ff5-4e01-8767-68d629b125e2,DISK], DatanodeInfoWithStorage[127.0.0.1:34717,DS-15daf992-9329-4b25-9667-a096fb2deda0,DISK]] 2023-07-18 04:54:43,740 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:43,740 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-18 04:54:43,740 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-18 04:54:43,740 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-18 04:54:43,740 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-18 04:54:43,740 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:43,741 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-18 04:54:43,741 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-18 04:54:43,742 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-18 04:54:43,743 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/info 2023-07-18 04:54:43,743 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/info 2023-07-18 04:54:43,744 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-18 04:54:43,748 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:43,750 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/info/97ba85398a9b4013bfdbe13aef0066c8 2023-07-18 04:54:43,750 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:43,751 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-18 04:54:43,752 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/rep_barrier 2023-07-18 04:54:43,752 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/rep_barrier 2023-07-18 04:54:43,752 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-18 04:54:43,759 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/rep_barrier/d9ffd0a2214e4da78f73687fcca695d9 2023-07-18 04:54:43,759 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:43,759 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-18 04:54:43,760 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/table 2023-07-18 04:54:43,760 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/table 2023-07-18 04:54:43,761 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-18 04:54:43,767 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/table/cb85068bae134b3cb32bf67c6cb9911e 2023-07-18 04:54:43,767 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:43,767 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740 2023-07-18 04:54:43,768 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740 2023-07-18 04:54:43,771 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-18 04:54:43,773 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-18 04:54:43,773 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61786799, jitterRate=-0.07930494844913483}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-18 04:54:43,774 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-18 04:54:43,776 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=21, masterSystemTime=1689656083721 2023-07-18 04:54:43,783 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-18 04:54:43,784 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-18 04:54:43,784 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45615,1689656083190, state=OPEN 2023-07-18 04:54:43,788 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-18 04:54:43,788 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-18 04:54:43,790 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-07-18 04:54:43,790 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45615,1689656083190 in 225 msec 2023-07-18 04:54:43,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=18 2023-07-18 04:54:43,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 385 msec 2023-07-18 04:54:43,854 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:43,960 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:44,065 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:44,171 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:44,277 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:44,382 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:44,488 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:44,593 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:44,699 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:44,805 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:44,910 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:44,952 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1552ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-07-18 04:54:45,016 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:45,121 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:45,226 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:45,332 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:45,437 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:45,542 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:45,649 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:45,755 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:45,861 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:45,967 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:46,073 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:46,178 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:46,284 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:46,389 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:46,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3055ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-18 04:54:46,495 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:46,600 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:46,706 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:46,748 INFO [jenkins-hbase4:37265Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 04:54:46,748 INFO [jenkins-hbase4:37265Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-18 04:54:46,811 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:46,919 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:47,024 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:47,129 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:47,235 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:47,341 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:47,418 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 04:54:47,446 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:47,552 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:47,657 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:47,763 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:47,868 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:47,907 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-18 04:54:47,907 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-18 04:54:47,912 DEBUG [hconnection-0x4fe96bcc-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:47,915 INFO [RS-EventLoopGroup-14-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58082, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:47,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=8750ae11f1669b3efa9c8fcc2b1be9cf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,45857,1689656054144, regionLocation=jenkins-hbase4.apache.org,45857,1689656054144, openSeqNum=2 2023-07-18 04:54:47,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=db11400df50c3fb2fe7bab3791162e0a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,45857,1689656054144, regionLocation=jenkins-hbase4.apache.org,45857,1689656054144, openSeqNum=2 2023-07-18 04:54:47,921 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=7f6a4e55b8779eead03ae75b46b98c60, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,45857,1689656054144, regionLocation=jenkins-hbase4.apache.org,45857,1689656054144, openSeqNum=2 2023-07-18 04:54:47,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-18 04:54:47,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689656147921 2023-07-18 04:54:47,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689656207921 2023-07-18 04:54:47,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 13 msec 2023-07-18 04:54:47,940 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,42175,1689656054096 had 1 regions 2023-07-18 04:54:47,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36243,1689656083152-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:47,940 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,45857,1689656054144 had 3 regions 2023-07-18 04:54:47,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36243,1689656083152-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:47,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36243,1689656083152-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:47,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:36243, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:47,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:47,940 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. is NOT online; state={8750ae11f1669b3efa9c8fcc2b1be9cf state=OPEN, ts=1689656087920, server=jenkins-hbase4.apache.org,45857,1689656054144}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-18 04:54:47,942 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,42175,1689656054096, splitWal=true, meta=true, isMeta: false 2023-07-18 04:54:47,942 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45857,1689656054144, splitWal=true, meta=false, isMeta: false 2023-07-18 04:54:47,944 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,45857,1689656054144/hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf., unknown_server=jenkins-hbase4.apache.org,45857,1689656054144/t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a., unknown_server=jenkins-hbase4.apache.org,45857,1689656054144/t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:47,945 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096-splitting dir is empty, no logs to split. 2023-07-18 04:54:47,945 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,42175,1689656054096 WAL count=0, meta=false 2023-07-18 04:54:47,945 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144-splitting 2023-07-18 04:54:47,946 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144-splitting dir is empty, no logs to split. 2023-07-18 04:54:47,946 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,45857,1689656054144 WAL count=0, meta=false 2023-07-18 04:54:47,948 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,42175,1689656054096-splitting dir is empty, no logs to split. 2023-07-18 04:54:47,948 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,42175,1689656054096 WAL count=0, meta=false 2023-07-18 04:54:47,948 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,42175,1689656054096 WAL splitting is done? wals=0, meta=false 2023-07-18 04:54:47,949 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144-splitting dir is empty, no logs to split. 2023-07-18 04:54:47,949 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,45857,1689656054144 WAL count=0, meta=false 2023-07-18 04:54:47,949 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,45857,1689656054144 WAL splitting is done? wals=0, meta=false 2023-07-18 04:54:47,950 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,42175,1689656054096 after splitting done 2023-07-18 04:54:47,950 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,42175,1689656054096 from processing; numProcessing=1 2023-07-18 04:54:47,952 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,45857,1689656054144 failed, ignore...File hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45857,1689656054144-splitting does not exist. 2023-07-18 04:54:47,952 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,42175,1689656054096, splitWal=true, meta=true in 4.6060 sec 2023-07-18 04:54:47,953 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=8750ae11f1669b3efa9c8fcc2b1be9cf, ASSIGN}, {pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=7f6a4e55b8779eead03ae75b46b98c60, ASSIGN}, {pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=db11400df50c3fb2fe7bab3791162e0a, ASSIGN}] 2023-07-18 04:54:47,954 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=8750ae11f1669b3efa9c8fcc2b1be9cf, ASSIGN 2023-07-18 04:54:47,954 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=7f6a4e55b8779eead03ae75b46b98c60, ASSIGN 2023-07-18 04:54:47,954 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=db11400df50c3fb2fe7bab3791162e0a, ASSIGN 2023-07-18 04:54:47,955 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=8750ae11f1669b3efa9c8fcc2b1be9cf, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 04:54:47,955 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=7f6a4e55b8779eead03ae75b46b98c60, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 04:54:47,955 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=db11400df50c3fb2fe7bab3791162e0a, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 04:54:47,973 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:48,079 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:48,105 DEBUG [jenkins-hbase4:36243] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-18 04:54:48,109 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=8750ae11f1669b3efa9c8fcc2b1be9cf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:48,109 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=db11400df50c3fb2fe7bab3791162e0a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:48,109 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=7f6a4e55b8779eead03ae75b46b98c60, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:48,109 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689656088108"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656088108"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656088108"}]},"ts":"1689656088108"} 2023-07-18 04:54:48,110 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656088109"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656088109"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656088109"}]},"ts":"1689656088109"} 2023-07-18 04:54:48,110 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656088109"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656088109"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656088109"}]},"ts":"1689656088109"} 2023-07-18 04:54:48,113 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure 8750ae11f1669b3efa9c8fcc2b1be9cf, server=jenkins-hbase4.apache.org,45615,1689656083190}] 2023-07-18 04:54:48,114 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=24, state=RUNNABLE; OpenRegionProcedure db11400df50c3fb2fe7bab3791162e0a, server=jenkins-hbase4.apache.org,45615,1689656083190}] 2023-07-18 04:54:48,115 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=23, state=RUNNABLE; OpenRegionProcedure 7f6a4e55b8779eead03ae75b46b98c60, server=jenkins-hbase4.apache.org,45615,1689656083190}] 2023-07-18 04:54:48,185 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:48,270 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:48,270 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:48,272 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1, suffix=, logDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190, archiveDir=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs, maxLogs=10 2023-07-18 04:54:48,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:48,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => db11400df50c3fb2fe7bab3791162e0a, NAME => 't1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:48,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:48,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:48,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:48,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:48,279 INFO [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:48,280 DEBUG [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/cf1 2023-07-18 04:54:48,280 DEBUG [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/cf1 2023-07-18 04:54:48,280 INFO [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db11400df50c3fb2fe7bab3791162e0a columnFamilyName cf1 2023-07-18 04:54:48,291 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1 for peer 1 2023-07-18 04:54:48,292 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190/jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1.1689656088273 2023-07-18 04:54:48,292 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1 2023-07-18 04:54:48,293 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 04:54:48,293 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:48,294 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e88dc907a1be4e2fb34eb8d7b6bd8b66 2023-07-18 04:54:48,294 DEBUG [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] regionserver.HStore(539): loaded hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/cf1/e88dc907a1be4e2fb34eb8d7b6bd8b66 2023-07-18 04:54:48,294 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36951,DS-63fc3a07-0ff5-4e01-8767-68d629b125e2,DISK], DatanodeInfoWithStorage[127.0.0.1:34717,DS-15daf992-9329-4b25-9667-a096fb2deda0,DISK]] 2023-07-18 04:54:48,295 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1 2023-07-18 04:54:48,295 INFO [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] regionserver.HStore(310): Store=db11400df50c3fb2fe7bab3791162e0a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:48,295 INFO [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:48,295 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8750ae11f1669b3efa9c8fcc2b1be9cf, NAME => 'hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:48,295 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:48,295 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:48,295 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:48,295 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:48,297 DEBUG [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/norep 2023-07-18 04:54:48,297 DEBUG [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/norep 2023-07-18 04:54:48,297 INFO [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db11400df50c3fb2fe7bab3791162e0a columnFamilyName norep 2023-07-18 04:54:48,297 INFO [StoreOpener-8750ae11f1669b3efa9c8fcc2b1be9cf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:48,299 DEBUG [StoreOpener-8750ae11f1669b3efa9c8fcc2b1be9cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf/info 2023-07-18 04:54:48,299 DEBUG [StoreOpener-8750ae11f1669b3efa9c8fcc2b1be9cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf/info 2023-07-18 04:54:48,299 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-18 04:54:48,300 INFO [StoreOpener-8750ae11f1669b3efa9c8fcc2b1be9cf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8750ae11f1669b3efa9c8fcc2b1be9cf columnFamilyName info 2023-07-18 04:54:48,305 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6cf2207f2f674243be9f1b7a2f68549b 2023-07-18 04:54:48,305 DEBUG [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] regionserver.HStore(539): loaded hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/norep/6cf2207f2f674243be9f1b7a2f68549b 2023-07-18 04:54:48,305 INFO [StoreOpener-db11400df50c3fb2fe7bab3791162e0a-1] regionserver.HStore(310): Store=db11400df50c3fb2fe7bab3791162e0a/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:48,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:48,307 DEBUG [StoreOpener-8750ae11f1669b3efa9c8fcc2b1be9cf-1] regionserver.HStore(539): loaded hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf/info/150be18a83f248079f73a2772765d7c9 2023-07-18 04:54:48,308 INFO [StoreOpener-8750ae11f1669b3efa9c8fcc2b1be9cf-1] regionserver.HStore(310): Store=8750ae11f1669b3efa9c8fcc2b1be9cf/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:48,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:48,308 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:48,310 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:48,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 04:54:48,312 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:48,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened db11400df50c3fb2fe7bab3791162e0a; next sequenceid=110; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67151976, jitterRate=6.424188613891602E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 04:54:48,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for db11400df50c3fb2fe7bab3791162e0a: 2023-07-18 04:54:48,316 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:48,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a., pid=26, masterSystemTime=1689656088266 2023-07-18 04:54:48,317 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8750ae11f1669b3efa9c8fcc2b1be9cf; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67954511, jitterRate=0.012601122260093689}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 04:54:48,317 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8750ae11f1669b3efa9c8fcc2b1be9cf: 2023-07-18 04:54:48,322 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf., pid=25, masterSystemTime=1689656088266 2023-07-18 04:54:48,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:48,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:48,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:48,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7f6a4e55b8779eead03ae75b46b98c60, NAME => 't2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:48,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:48,324 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=db11400df50c3fb2fe7bab3791162e0a, regionState=OPEN, repBarrier=110, openSeqNum=110, regionLocation=jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:48,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:48,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:48,324 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656088324"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656088324"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656088324"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656088324"}]},"ts":"1689656088324"} 2023-07-18 04:54:48,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:48,325 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:48,325 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:48,325 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=8750ae11f1669b3efa9c8fcc2b1be9cf, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:48,326 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689656088325"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656088325"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656088325"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656088325"}]},"ts":"1689656088325"} 2023-07-18 04:54:48,327 INFO [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:48,328 DEBUG [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/cf1 2023-07-18 04:54:48,328 DEBUG [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/cf1 2023-07-18 04:54:48,329 INFO [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7f6a4e55b8779eead03ae75b46b98c60 columnFamilyName cf1 2023-07-18 04:54:48,331 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=24 2023-07-18 04:54:48,331 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=24, state=SUCCESS; OpenRegionProcedure db11400df50c3fb2fe7bab3791162e0a, server=jenkins-hbase4.apache.org,45615,1689656083190 in 212 msec 2023-07-18 04:54:48,332 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-07-18 04:54:48,332 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure 8750ae11f1669b3efa9c8fcc2b1be9cf, server=jenkins-hbase4.apache.org,45615,1689656083190 in 214 msec 2023-07-18 04:54:48,332 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=db11400df50c3fb2fe7bab3791162e0a, ASSIGN in 378 msec 2023-07-18 04:54:48,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=8750ae11f1669b3efa9c8fcc2b1be9cf, ASSIGN in 379 msec 2023-07-18 04:54:48,337 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6e0003805a0f4f4c884b382a37632a83 2023-07-18 04:54:48,337 DEBUG [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] regionserver.HStore(539): loaded hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/cf1/6e0003805a0f4f4c884b382a37632a83 2023-07-18 04:54:48,337 INFO [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] regionserver.HStore(310): Store=7f6a4e55b8779eead03ae75b46b98c60/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:48,337 INFO [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:48,338 DEBUG [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/norep 2023-07-18 04:54:48,338 DEBUG [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/norep 2023-07-18 04:54:48,339 INFO [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7f6a4e55b8779eead03ae75b46b98c60 columnFamilyName norep 2023-07-18 04:54:48,347 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27c15cac5541443d89a7c280b70cebb3 2023-07-18 04:54:48,348 DEBUG [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] regionserver.HStore(539): loaded hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/norep/27c15cac5541443d89a7c280b70cebb3 2023-07-18 04:54:48,348 INFO [StoreOpener-7f6a4e55b8779eead03ae75b46b98c60-1] regionserver.HStore(310): Store=7f6a4e55b8779eead03ae75b46b98c60/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:48,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:48,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:48,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 04:54:48,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:48,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7f6a4e55b8779eead03ae75b46b98c60; next sequenceid=210; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63066721, jitterRate=-0.06023262441158295}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 04:54:48,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7f6a4e55b8779eead03ae75b46b98c60: 2023-07-18 04:54:48,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60., pid=27, masterSystemTime=1689656088266 2023-07-18 04:54:48,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:48,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:48,358 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=7f6a4e55b8779eead03ae75b46b98c60, regionState=OPEN, repBarrier=210, openSeqNum=210, regionLocation=jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:48,359 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656088358"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656088358"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656088358"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656088358"}]},"ts":"1689656088358"} 2023-07-18 04:54:48,363 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=23 2023-07-18 04:54:48,363 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=23, state=SUCCESS; OpenRegionProcedure 7f6a4e55b8779eead03ae75b46b98c60, server=jenkins-hbase4.apache.org,45615,1689656083190 in 246 msec 2023-07-18 04:54:48,365 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=19 2023-07-18 04:54:48,365 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,45857,1689656054144 after splitting done 2023-07-18 04:54:48,365 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=7f6a4e55b8779eead03ae75b46b98c60, ASSIGN in 410 msec 2023-07-18 04:54:48,366 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,45857,1689656054144 from processing; numProcessing=0 2023-07-18 04:54:48,367 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,45857,1689656054144, splitWal=true, meta=false in 5.0180 sec 2023-07-18 04:54:48,400 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-18 04:54:48,400 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 04:54:48,400 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-18 04:54:48,400 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190/jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0.1689656083619, lastWalPosition=452, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-18 04:54:48,404 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 793 2023-07-18 04:54:48,404 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 04:54:48,404 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 793 2023-07-18 04:54:48,404 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190/jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1.1689656088273, lastWalPosition=793, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-18 04:54:48,506 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-18 04:54:48,509 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 793 2023-07-18 04:54:48,513 INFO [jenkins-hbase4:45615Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0]: currently replicating from: hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190/jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0.1689656083619 at position: 452 walGroup [jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1]: currently replicating from: hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190/jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1.1689656088273 at position: 793 2023-07-18 04:54:48,611 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-18 04:54:48,614 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 793 2023-07-18 04:54:48,718 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-18 04:54:48,719 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 793 2023-07-18 04:54:48,824 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 793 2023-07-18 04:54:48,824 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-18 04:54:48,930 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-18 04:54:48,931 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 793 2023-07-18 04:54:48,941 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on existing znode=/1/namespace 2023-07-18 04:54:48,954 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-18 04:54:48,957 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-18 04:54:48,957 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.735sec 2023-07-18 04:54:48,957 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-18 04:54:48,957 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-18 04:54:48,957 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-18 04:54:48,957 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36243,1689656083152-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-18 04:54:48,957 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36243,1689656083152-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-18 04:54:48,961 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-18 04:54:49,018 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(139): Connect 0x05058a68 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:49,022 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@540926e0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:49,024 DEBUG [hconnection-0x7de13613-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:49,026 INFO [RS-EventLoopGroup-14-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58090, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:49,031 INFO [Listener at localhost/37011] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-18 04:54:49,031 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x05058a68 to 127.0.0.1:63236 2023-07-18 04:54:49,031 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:49,033 INFO [Listener at localhost/37011] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:36243 after: jenkins-hbase4.apache.org:36243 2023-07-18 04:54:49,033 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(139): Connect 0x4407444d to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:49,038 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@30a2a370, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:49,038 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-18 04:54:49,038 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4474cedf to 127.0.0.1:63236 2023-07-18 04:54:49,038 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:49,038 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 793 2023-07-18 04:54:49,038 DEBUG [Listener at localhost/37011] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-18 04:54:49,038 DEBUG [Listener at localhost/37011] util.JVMClusterUtil(257): Found active master hash=812371474, stopped=false 2023-07-18 04:54:49,039 INFO [Listener at localhost/37011] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33929,1689656066387 2023-07-18 04:54:49,041 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:37265-0x10176c7bf3c001d, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 04:54:49,041 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 04:54:49,041 INFO [Listener at localhost/37011] procedure2.ProcedureExecutor(629): Stopping 2023-07-18 04:54:49,041 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:49,043 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x60b3ad3e to 127.0.0.1:63236 2023-07-18 04:54:49,044 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:49,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:49,044 INFO [Listener at localhost/37011] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,37265,1689656066415' ***** 2023-07-18 04:54:49,044 INFO [Listener at localhost/37011] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 04:54:49,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37265-0x10176c7bf3c001d, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:49,044 INFO [RS:0;jenkins-hbase4:37265] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 04:54:49,044 INFO [RS:0;jenkins-hbase4:37265] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 04:54:49,044 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 04:54:49,044 INFO [RS:0;jenkins-hbase4:37265] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 04:54:49,045 INFO [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(3305): Received CLOSE for a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:49,046 INFO [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(3305): Received CLOSE for 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:49,046 INFO [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(3305): Received CLOSE for 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:49,046 INFO [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:49,046 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a2e71ceab1ce1a583f43e863ff7cd0a0, disabling compactions & flushes 2023-07-18 04:54:49,046 DEBUG [RS:0;jenkins-hbase4:37265] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0c3bf1b5 to 127.0.0.1:63236 2023-07-18 04:54:49,046 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:49,046 DEBUG [RS:0;jenkins-hbase4:37265] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:49,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:49,047 INFO [RS:0;jenkins-hbase4:37265] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 04:54:49,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. after waiting 0 ms 2023-07-18 04:54:49,047 INFO [RS:0;jenkins-hbase4:37265] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 04:54:49,047 INFO [RS:0;jenkins-hbase4:37265] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 04:54:49,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:49,047 INFO [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-18 04:54:49,047 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing a2e71ceab1ce1a583f43e863ff7cd0a0 2/2 column families, dataSize=1.36 KB heapSize=5.58 KB 2023-07-18 04:54:49,047 INFO [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-18 04:54:49,047 DEBUG [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(1478): Online Regions={a2e71ceab1ce1a583f43e863ff7cd0a0=t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0., 1588230740=hbase:meta,,1.1588230740, 1b9326fa16ae8497bb1df12a88dfd577=t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577., 383bb30520997d1bb825dc5e5f31999e=hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.} 2023-07-18 04:54:49,048 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-18 04:54:49,048 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:49,048 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-18 04:54:49,049 DEBUG [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(1504): Waiting on 1588230740, 1b9326fa16ae8497bb1df12a88dfd577, 383bb30520997d1bb825dc5e5f31999e, a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:49,049 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-18 04:54:49,049 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-18 04:54:49,049 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-18 04:54:49,049 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-18 04:54:49,069 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.36 KB at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/.tmp/cf1/d6a832002bf544daad6d8b1c1648d003 2023-07-18 04:54:49,073 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/.tmp/info/2e79da83316e4b4cb208a78c9bd67186 2023-07-18 04:54:49,076 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d6a832002bf544daad6d8b1c1648d003 2023-07-18 04:54:49,077 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/.tmp/cf1/d6a832002bf544daad6d8b1c1648d003 as hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1/d6a832002bf544daad6d8b1c1648d003 2023-07-18 04:54:49,081 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/.tmp/info/2e79da83316e4b4cb208a78c9bd67186 as hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info/2e79da83316e4b4cb208a78c9bd67186 2023-07-18 04:54:49,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d6a832002bf544daad6d8b1c1648d003 2023-07-18 04:54:49,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1/d6a832002bf544daad6d8b1c1648d003, entries=50, sequenceid=14, filesize=6.4 K 2023-07-18 04:54:49,085 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.36 KB/1390, heapSize ~5.31 KB/5440, currentSize=0 B/0 for a2e71ceab1ce1a583f43e863ff7cd0a0 in 38ms, sequenceid=14, compaction requested=false 2023-07-18 04:54:49,093 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info/2e79da83316e4b4cb208a78c9bd67186, entries=24, sequenceid=31, filesize=7.4 K 2023-07-18 04:54:49,094 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 45ms, sequenceid=31, compaction requested=false 2023-07-18 04:54:49,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=10 2023-07-18 04:54:49,106 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:49,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a2e71ceab1ce1a583f43e863ff7cd0a0: 2023-07-18 04:54:49,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:49,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1b9326fa16ae8497bb1df12a88dfd577, disabling compactions & flushes 2023-07-18 04:54:49,107 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:49,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:49,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. after waiting 0 ms 2023-07-18 04:54:49,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:49,108 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1b9326fa16ae8497bb1df12a88dfd577 2/2 column families, dataSize=2.72 KB heapSize=10.66 KB 2023-07-18 04:54:49,109 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-18 04:54:49,110 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-18 04:54:49,111 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-18 04:54:49,111 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-18 04:54:49,111 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-18 04:54:49,125 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=15 (bloomFilter=true), to=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/.tmp/cf1/e685f233fda344e68e26feddcadc4b60 2023-07-18 04:54:49,130 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e685f233fda344e68e26feddcadc4b60 2023-07-18 04:54:49,130 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/.tmp/cf1/e685f233fda344e68e26feddcadc4b60 as hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1/e685f233fda344e68e26feddcadc4b60 2023-07-18 04:54:49,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e685f233fda344e68e26feddcadc4b60 2023-07-18 04:54:49,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1/e685f233fda344e68e26feddcadc4b60, entries=100, sequenceid=15, filesize=7.9 K 2023-07-18 04:54:49,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2790, heapSize ~10.39 KB/10640, currentSize=0 B/0 for 1b9326fa16ae8497bb1df12a88dfd577 in 32ms, sequenceid=15, compaction requested=false 2023-07-18 04:54:49,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/recovered.edits/18.seqid, newMaxSeqId=18, maxSeqId=11 2023-07-18 04:54:49,150 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-18 04:54:49,150 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 793 2023-07-18 04:54:49,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:49,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1b9326fa16ae8497bb1df12a88dfd577: 2023-07-18 04:54:49,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:49,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 383bb30520997d1bb825dc5e5f31999e, disabling compactions & flushes 2023-07-18 04:54:49,152 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:49,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:49,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. after waiting 0 ms 2023-07-18 04:54:49,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:49,156 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-18 04:54:49,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:49,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 383bb30520997d1bb825dc5e5f31999e: 2023-07-18 04:54:49,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:49,249 INFO [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,37265,1689656066415; all regions closed. 2023-07-18 04:54:49,249 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:49,260 DEBUG [RS:0;jenkins-hbase4:37265] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs 2023-07-18 04:54:49,260 INFO [RS:0;jenkins-hbase4:37265] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C37265%2C1689656066415.meta:.meta(num 1689656066971) 2023-07-18 04:54:49,261 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:49,261 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-18 04:54:49,261 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 793 2023-07-18 04:54:49,271 DEBUG [RS:0;jenkins-hbase4:37265] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs 2023-07-18 04:54:49,271 INFO [RS:0;jenkins-hbase4:37265] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C37265%2C1689656066415.jenkins-hbase4.apache.org%2C37265%2C1689656066415.regiongroup-0:(num 1689656066855) 2023-07-18 04:54:49,271 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:49,285 DEBUG [RS:0;jenkins-hbase4:37265] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs 2023-07-18 04:54:49,285 INFO [RS:0;jenkins-hbase4:37265] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C37265%2C1689656066415.jenkins-hbase4.apache.org%2C37265%2C1689656066415.regiongroup-1:(num 1689656071517) 2023-07-18 04:54:49,285 DEBUG [RS:0;jenkins-hbase4:37265] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:49,285 INFO [RS:0;jenkins-hbase4:37265] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:49,285 INFO [RS:0;jenkins-hbase4:37265] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-18 04:54:49,285 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 04:54:49,286 DEBUG [RS:0;jenkins-hbase4:37265] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6416e408 to 127.0.0.1:63236 2023-07-18 04:54:49,286 DEBUG [RS:0;jenkins-hbase4:37265] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:49,287 INFO [RS:0;jenkins-hbase4:37265] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37265 2023-07-18 04:54:49,292 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:37265-0x10176c7bf3c001d, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,37265,1689656066415 2023-07-18 04:54:49,292 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:49,292 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:37265-0x10176c7bf3c001d, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:49,292 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10176c7bf3c0026, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:49,292 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 04:54:49,293 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,37265,1689656066415] 2023-07-18 04:54:49,293 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37265,1689656066415; numProcessing=1 2023-07-18 04:54:49,294 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,37265,1689656066415 already deleted, retry=false 2023-07-18 04:54:49,294 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,37265,1689656066415 expired; onlineServers=0 2023-07-18 04:54:49,294 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33929,1689656066387' ***** 2023-07-18 04:54:49,294 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-18 04:54:49,295 DEBUG [M:0;jenkins-hbase4:33929] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@34545cc7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 04:54:49,295 INFO [M:0;jenkins-hbase4:33929] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33929,1689656066387 2023-07-18 04:54:49,295 INFO [M:0;jenkins-hbase4:33929] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33929,1689656066387; all regions closed. 2023-07-18 04:54:49,295 DEBUG [M:0;jenkins-hbase4:33929] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:49,295 DEBUG [M:0;jenkins-hbase4:33929] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-18 04:54:49,295 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-18 04:54:49,295 DEBUG [M:0;jenkins-hbase4:33929] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-18 04:54:49,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656066624] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656066624,5,FailOnTimeoutGroup] 2023-07-18 04:54:49,296 INFO [M:0;jenkins-hbase4:33929] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-18 04:54:49,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656066624] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656066624,5,FailOnTimeoutGroup] 2023-07-18 04:54:49,296 INFO [M:0;jenkins-hbase4:33929] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-18 04:54:49,297 INFO [M:0;jenkins-hbase4:33929] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-18 04:54:49,297 DEBUG [M:0;jenkins-hbase4:33929] master.HMaster(1512): Stopping service threads 2023-07-18 04:54:49,297 INFO [M:0;jenkins-hbase4:33929] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-18 04:54:49,298 ERROR [M:0;jenkins-hbase4:33929] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-18 04:54:49,298 INFO [M:0;jenkins-hbase4:33929] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-18 04:54:49,298 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-18 04:54:49,298 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-18 04:54:49,299 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:49,299 DEBUG [M:0;jenkins-hbase4:33929] zookeeper.ZKUtil(398): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-18 04:54:49,299 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 04:54:49,299 WARN [M:0;jenkins-hbase4:33929] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-18 04:54:49,299 INFO [M:0;jenkins-hbase4:33929] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-18 04:54:49,300 INFO [M:0;jenkins-hbase4:33929] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-18 04:54:49,300 DEBUG [M:0;jenkins-hbase4:33929] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-18 04:54:49,300 INFO [M:0;jenkins-hbase4:33929] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:49,301 DEBUG [M:0;jenkins-hbase4:33929] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:49,301 DEBUG [M:0;jenkins-hbase4:33929] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-18 04:54:49,301 DEBUG [M:0;jenkins-hbase4:33929] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:49,301 INFO [M:0;jenkins-hbase4:33929] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=35.35 KB heapSize=44.01 KB 2023-07-18 04:54:49,374 INFO [M:0;jenkins-hbase4:33929] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=35.35 KB at sequenceid=217 (bloomFilter=true), to=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e1b8c52d3f0c47a3b53f87b774c972d5 2023-07-18 04:54:49,380 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-18 04:54:49,380 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 793 2023-07-18 04:54:49,382 DEBUG [M:0;jenkins-hbase4:33929] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e1b8c52d3f0c47a3b53f87b774c972d5 as hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e1b8c52d3f0c47a3b53f87b774c972d5 2023-07-18 04:54:49,387 INFO [M:0;jenkins-hbase4:33929] regionserver.HStore(1080): Added hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e1b8c52d3f0c47a3b53f87b774c972d5, entries=12, sequenceid=217, filesize=6.4 K 2023-07-18 04:54:49,388 INFO [M:0;jenkins-hbase4:33929] regionserver.HRegion(2948): Finished flush of dataSize ~35.35 KB/36201, heapSize ~43.99 KB/45048, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 87ms, sequenceid=217, compaction requested=false 2023-07-18 04:54:49,391 INFO [M:0;jenkins-hbase4:33929] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:49,391 DEBUG [M:0;jenkins-hbase4:33929] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 04:54:49,393 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,33929,1689656066387 2023-07-18 04:54:49,401 INFO [M:0;jenkins-hbase4:33929] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-18 04:54:49,401 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 04:54:49,401 INFO [M:0;jenkins-hbase4:33929] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33929 2023-07-18 04:54:49,405 DEBUG [M:0;jenkins-hbase4:33929] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,33929,1689656066387 already deleted, retry=false 2023-07-18 04:54:49,486 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-18 04:54:49,487 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 793 2023-07-18 04:54:49,554 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:49,554 INFO [M:0;jenkins-hbase4:33929] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33929,1689656066387; zookeeper connection closed. 2023-07-18 04:54:49,554 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:33929-0x10176c7bf3c001c, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:49,592 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 452 2023-07-18 04:54:49,592 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 793 2023-07-18 04:54:49,654 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:37265-0x10176c7bf3c001d, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:49,654 INFO [RS:0;jenkins-hbase4:37265] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,37265,1689656066415; zookeeper connection closed. 2023-07-18 04:54:49,654 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:37265-0x10176c7bf3c001d, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:49,655 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5329315a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5329315a 2023-07-18 04:54:49,655 INFO [Listener at localhost/37011] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-18 04:54:49,657 DEBUG [hconnection-0x1e7f3317-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:49,661 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58094, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:49,697 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 793 2023-07-18 04:54:49,700 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 3384 2023-07-18 04:54:49,700 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 28 WAL entries eligible for replication 2023-07-18 04:54:49,700 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 3384 2023-07-18 04:54:49,700 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/db11400df50c3fb2fe7bab3791162e0a/112=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/113=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/114=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/115=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/116=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/117=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/118=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/119=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/120=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/121=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/122=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/123=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/124=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/125=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/126=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/127=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/128=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/129=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/130=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/131=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/132=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/133=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/134=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/135=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/136=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/137=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/138=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/139=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190/jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0.1689656083619, lastWalPosition=3384, nbRowKeys=28, nbHFiles=0, heapSize=5824, lastSeqIds={}, endOfFile=false,usedBufferSize=5824] 2023-07-18 04:54:49,703 WARN [RS:0;jenkins-hbase4:45615.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 04:54:49,803 WARN [RS:0;jenkins-hbase4:45615.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 04:54:49,804 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 2681 2023-07-18 04:54:49,804 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 18 WAL entries eligible for replication 2023-07-18 04:54:49,804 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/212=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/213=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/214=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/215=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/216=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/217=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/218=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/219=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/220=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/221=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/222=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/223=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/224=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/225=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/226=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/227=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/228=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/229=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190/jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1.1689656088273, lastWalPosition=2681, nbRowKeys=18, nbHFiles=0, heapSize=3744, lastSeqIds={}, endOfFile=false,usedBufferSize=3744] 2023-07-18 04:54:49,805 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 2681 2023-07-18 04:54:49,807 WARN [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 04:54:49,815 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 11128 2023-07-18 04:54:49,815 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 72 WAL entries eligible for replication 2023-07-18 04:54:49,815 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 11128 2023-07-18 04:54:49,907 WARN [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 04:54:49,917 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 13965 2023-07-18 04:54:49,918 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 106 WAL entries eligible for replication 2023-07-18 04:54:49,918 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 13965 2023-07-18 04:54:49,921 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 11128 2023-07-18 04:54:50,001 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4407444d to 127.0.0.1:63236 2023-07-18 04:54:50,001 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:50,001 DEBUG [Listener at localhost/37011] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-18 04:54:50,001 DEBUG [Listener at localhost/37011] util.JVMClusterUtil(257): Found active master hash=1103901817, stopped=false 2023-07-18 04:54:50,001 INFO [Listener at localhost/37011] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,36243,1689656083152 2023-07-18 04:54:50,004 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:45615-0x10176c7bf3c0029, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-18 04:54:50,004 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-18 04:54:50,004 WARN [RS:0;jenkins-hbase4:45615.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 04:54:50,004 INFO [Listener at localhost/37011] procedure2.ProcedureExecutor(629): Stopping 2023-07-18 04:54:50,004 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:50,006 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45615-0x10176c7bf3c0029, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 04:54:50,006 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-18 04:54:50,006 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3aa5c70f to 127.0.0.1:63236 2023-07-18 04:54:50,006 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:50,006 INFO [Listener at localhost/37011] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45615,1689656083190' ***** 2023-07-18 04:54:50,007 INFO [Listener at localhost/37011] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 04:54:50,007 INFO [RS:0;jenkins-hbase4:45615] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 04:54:50,007 INFO [RS:0;jenkins-hbase4:45615] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 04:54:50,007 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 04:54:50,007 INFO [RS:0;jenkins-hbase4:45615] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 04:54:50,008 INFO [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(3305): Received CLOSE for db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:50,008 INFO [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(3305): Received CLOSE for 8750ae11f1669b3efa9c8fcc2b1be9cf 2023-07-18 04:54:50,008 INFO [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(3305): Received CLOSE for 7f6a4e55b8779eead03ae75b46b98c60 2023-07-18 04:54:50,008 INFO [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:50,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing db11400df50c3fb2fe7bab3791162e0a, disabling compactions & flushes 2023-07-18 04:54:50,009 DEBUG [RS:0;jenkins-hbase4:45615] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x77d0ac0d to 127.0.0.1:63236 2023-07-18 04:54:50,009 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:50,009 DEBUG [RS:0;jenkins-hbase4:45615] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:50,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:50,009 INFO [RS:0;jenkins-hbase4:45615] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 04:54:50,009 INFO [RS:0;jenkins-hbase4:45615] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 04:54:50,009 INFO [RS:0;jenkins-hbase4:45615] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 04:54:50,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. after waiting 0 ms 2023-07-18 04:54:50,009 INFO [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-18 04:54:50,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:50,010 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing db11400df50c3fb2fe7bab3791162e0a 2/2 column families, dataSize=3.44 KB heapSize=11.55 KB 2023-07-18 04:54:50,010 INFO [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-18 04:54:50,010 DEBUG [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, db11400df50c3fb2fe7bab3791162e0a=t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a., 8750ae11f1669b3efa9c8fcc2b1be9cf=hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf., 7f6a4e55b8779eead03ae75b46b98c60=t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60.} 2023-07-18 04:54:50,010 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-18 04:54:50,011 DEBUG [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(1504): Waiting on 1588230740, 7f6a4e55b8779eead03ae75b46b98c60, 8750ae11f1669b3efa9c8fcc2b1be9cf, db11400df50c3fb2fe7bab3791162e0a 2023-07-18 04:54:50,011 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-18 04:54:50,011 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-18 04:54:50,011 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-18 04:54:50,011 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-18 04:54:50,011 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.94 KB heapSize=5.62 KB 2023-07-18 04:54:50,029 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 22285 2023-07-18 04:54:50,030 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 76 WAL entries eligible for replication 2023-07-18 04:54:50,035 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/.tmp/cf1/75aa88046e15481a87fffe5bba1e4d22 2023-07-18 04:54:50,039 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 11371 2023-07-18 04:54:50,039 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-18 04:54:50,052 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/.tmp/norep/03b178838f1d46b7af5582459e458525 2023-07-18 04:54:50,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/.tmp/cf1/75aa88046e15481a87fffe5bba1e4d22 as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/cf1/75aa88046e15481a87fffe5bba1e4d22 2023-07-18 04:54:50,065 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/cf1/75aa88046e15481a87fffe5bba1e4d22, entries=100, sequenceid=213, filesize=8.4 K 2023-07-18 04:54:50,066 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/.tmp/norep/03b178838f1d46b7af5582459e458525 as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/norep/03b178838f1d46b7af5582459e458525 2023-07-18 04:54:50,073 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/norep/03b178838f1d46b7af5582459e458525, entries=1, sequenceid=213, filesize=4.8 K 2023-07-18 04:54:50,074 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.44 KB/3521, heapSize ~11.52 KB/11800, currentSize=0 B/0 for db11400df50c3fb2fe7bab3791162e0a in 64ms, sequenceid=213, compaction requested=false 2023-07-18 04:54:50,080 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:50,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t1_syncup/db11400df50c3fb2fe7bab3791162e0a/recovered.edits/216.seqid, newMaxSeqId=216, maxSeqId=109 2023-07-18 04:54:50,088 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:50,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for db11400df50c3fb2fe7bab3791162e0a: 2023-07-18 04:54:50,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689656059473.db11400df50c3fb2fe7bab3791162e0a. 2023-07-18 04:54:50,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8750ae11f1669b3efa9c8fcc2b1be9cf, disabling compactions & flushes 2023-07-18 04:54:50,089 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:50,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:50,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. after waiting 0 ms 2023-07-18 04:54:50,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:50,094 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/namespace/8750ae11f1669b3efa9c8fcc2b1be9cf/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-18 04:54:50,095 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:50,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8750ae11f1669b3efa9c8fcc2b1be9cf: 2023-07-18 04:54:50,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689656056015.8750ae11f1669b3efa9c8fcc2b1be9cf. 2023-07-18 04:54:50,096 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7f6a4e55b8779eead03ae75b46b98c60, disabling compactions & flushes 2023-07-18 04:54:50,096 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:50,096 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:50,096 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. after waiting 0 ms 2023-07-18 04:54:50,096 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:50,097 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 7f6a4e55b8779eead03ae75b46b98c60 2/2 column families, dataSize=7.05 KB heapSize=22.49 KB 2023-07-18 04:54:50,108 WARN [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 04:54:50,113 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/.tmp/cf1/6b7bc1f3fedc454b9c3346af764bf581 2023-07-18 04:54:50,128 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/.tmp/norep/986fdb5520664808b7ca7d56ab57950e 2023-07-18 04:54:50,133 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/.tmp/cf1/6b7bc1f3fedc454b9c3346af764bf581 as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/cf1/6b7bc1f3fedc454b9c3346af764bf581 2023-07-18 04:54:50,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/cf1/6b7bc1f3fedc454b9c3346af764bf581, entries=200, sequenceid=413, filesize=12.5 K 2023-07-18 04:54:50,140 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/.tmp/norep/986fdb5520664808b7ca7d56ab57950e as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/norep/986fdb5520664808b7ca7d56ab57950e 2023-07-18 04:54:50,145 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/norep/986fdb5520664808b7ca7d56ab57950e, entries=1, sequenceid=413, filesize=4.8 K 2023-07-18 04:54:50,146 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.05 KB/7221, heapSize ~22.46 KB/23000, currentSize=0 B/0 for 7f6a4e55b8779eead03ae75b46b98c60 in 50ms, sequenceid=413, compaction requested=false 2023-07-18 04:54:50,159 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/default/t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/recovered.edits/416.seqid, newMaxSeqId=416, maxSeqId=209 2023-07-18 04:54:50,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:50,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7f6a4e55b8779eead03ae75b46b98c60: 2023-07-18 04:54:50,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689656060116.7f6a4e55b8779eead03ae75b46b98c60. 2023-07-18 04:54:50,211 DEBUG [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-18 04:54:50,306 WARN [RS:0;jenkins-hbase4:45615.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 04:54:50,409 WARN [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-18 04:54:50,411 DEBUG [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-18 04:54:50,435 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/.tmp/info/a1b1ddbced87445bbc4c0b01423b0478 2023-07-18 04:54:50,449 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/.tmp/rep_barrier/f5f308ba441c48d09c7962c2f6f6f309 2023-07-18 04:54:50,455 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/.tmp/info/a1b1ddbced87445bbc4c0b01423b0478 as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/info/a1b1ddbced87445bbc4c0b01423b0478 2023-07-18 04:54:50,460 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/info/a1b1ddbced87445bbc4c0b01423b0478, entries=24, sequenceid=31, filesize=7.4 K 2023-07-18 04:54:50,461 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/.tmp/rep_barrier/f5f308ba441c48d09c7962c2f6f6f309 as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/rep_barrier/f5f308ba441c48d09c7962c2f6f6f309 2023-07-18 04:54:50,466 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/rep_barrier/f5f308ba441c48d09c7962c2f6f6f309, entries=2, sequenceid=31, filesize=4.9 K 2023-07-18 04:54:50,468 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.94 KB/3012, heapSize ~5.34 KB/5464, currentSize=0 B/0 for 1588230740 in 457ms, sequenceid=31, compaction requested=false 2023-07-18 04:54:50,475 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-18 04:54:50,475 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-18 04:54:50,479 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-18 04:54:50,479 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-18 04:54:50,479 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-18 04:54:50,479 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-18 04:54:50,480 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-18 04:54:50,611 INFO [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45615,1689656083190; all regions closed. 2023-07-18 04:54:50,612 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:50,618 DEBUG [RS:0;jenkins-hbase4:45615] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs 2023-07-18 04:54:50,619 INFO [RS:0;jenkins-hbase4:45615] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C45615%2C1689656083190.meta:.meta(num 1689656083730) 2023-07-18 04:54:50,619 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:50,626 DEBUG [RS:0;jenkins-hbase4:45615] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs 2023-07-18 04:54:50,626 INFO [RS:0;jenkins-hbase4:45615] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1:(num 1689656088273) 2023-07-18 04:54:50,626 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/WALs/jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:50,632 DEBUG [RS:0;jenkins-hbase4:45615] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs 2023-07-18 04:54:50,632 INFO [RS:0;jenkins-hbase4:45615] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0:(num 1689656083619) 2023-07-18 04:54:50,632 DEBUG [RS:0;jenkins-hbase4:45615] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:50,632 INFO [RS:0;jenkins-hbase4:45615] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:54:50,632 INFO [RS:0;jenkins-hbase4:45615] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-18 04:54:50,633 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 04:54:50,633 INFO [RS:0;jenkins-hbase4:45615] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-18 04:54:50,706 WARN [RS:0;jenkins-hbase4:45615.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1] regionserver.HBaseInterClusterReplicationEndpoint(515): No replication sinks found, returning without replicating. The source should retry with the same set of edits. Not logging this again for the next 10 seconds. 2023-07-18 04:54:50,735 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10176c7bf3c002d, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:50,735 DEBUG [RS:0;jenkins-hbase4:45615.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10176c7bf3c002d, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:50,735 DEBUG [RS:0;jenkins-hbase4:45615] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0d80e891 to 127.0.0.1:63236 2023-07-18 04:54:50,736 DEBUG [RS:0;jenkins-hbase4:45615] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:50,937 INFO [RS:0;jenkins-hbase4:45615] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS:0;jenkins-hbase4:45615.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1 terminated 2023-07-18 04:54:50,937 INFO [RS:0;jenkins-hbase4:45615] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1 terminated 2023-07-18 04:54:50,939 INFO [RS:0;jenkins-hbase4:45615] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45615 2023-07-18 04:54:50,944 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:45615-0x10176c7bf3c0029, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:54:50,944 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 04:54:50,944 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:45615-0x10176c7bf3c0029, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-18 04:54:50,946 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45615,1689656083190] 2023-07-18 04:54:50,946 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45615,1689656083190; numProcessing=1 2023-07-18 04:54:50,947 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,45615,1689656083190 already deleted, retry=false 2023-07-18 04:54:50,947 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45615,1689656083190 expired; onlineServers=0 2023-07-18 04:54:50,947 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36243,1689656083152' ***** 2023-07-18 04:54:50,947 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-18 04:54:50,948 DEBUG [M:0;jenkins-hbase4:36243] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@45f3a898, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 04:54:50,948 INFO [M:0;jenkins-hbase4:36243] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36243,1689656083152 2023-07-18 04:54:50,948 INFO [M:0;jenkins-hbase4:36243] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36243,1689656083152; all regions closed. 2023-07-18 04:54:50,948 DEBUG [M:0;jenkins-hbase4:36243] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:50,948 DEBUG [M:0;jenkins-hbase4:36243] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-18 04:54:50,948 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-18 04:54:50,948 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656083395] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656083395,5,FailOnTimeoutGroup] 2023-07-18 04:54:50,948 DEBUG [M:0;jenkins-hbase4:36243] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-18 04:54:50,948 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656083395] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656083395,5,FailOnTimeoutGroup] 2023-07-18 04:54:50,949 INFO [M:0;jenkins-hbase4:36243] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-18 04:54:50,950 INFO [M:0;jenkins-hbase4:36243] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-18 04:54:50,950 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-18 04:54:50,950 INFO [M:0;jenkins-hbase4:36243] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-18 04:54:50,950 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-18 04:54:50,950 DEBUG [M:0;jenkins-hbase4:36243] master.HMaster(1512): Stopping service threads 2023-07-18 04:54:50,950 INFO [M:0;jenkins-hbase4:36243] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-18 04:54:50,951 ERROR [M:0;jenkins-hbase4:36243] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-18 04:54:50,951 INFO [M:0;jenkins-hbase4:36243] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-18 04:54:50,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-18 04:54:50,951 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-18 04:54:50,951 DEBUG [M:0;jenkins-hbase4:36243] zookeeper.ZKUtil(398): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-18 04:54:50,951 WARN [M:0;jenkins-hbase4:36243] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-18 04:54:50,951 INFO [M:0;jenkins-hbase4:36243] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-18 04:54:50,952 INFO [M:0;jenkins-hbase4:36243] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-18 04:54:50,954 DEBUG [M:0;jenkins-hbase4:36243] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-18 04:54:50,954 INFO [M:0;jenkins-hbase4:36243] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:50,954 DEBUG [M:0;jenkins-hbase4:36243] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:50,954 DEBUG [M:0;jenkins-hbase4:36243] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-18 04:54:50,954 DEBUG [M:0;jenkins-hbase4:36243] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:50,954 INFO [M:0;jenkins-hbase4:36243] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=29.39 KB heapSize=36.55 KB 2023-07-18 04:54:50,964 INFO [M:0;jenkins-hbase4:36243] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=29.39 KB at sequenceid=217 (bloomFilter=true), to=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/591a7ecb679a4fe6b0f456e13bda44c5 2023-07-18 04:54:50,970 DEBUG [M:0;jenkins-hbase4:36243] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/591a7ecb679a4fe6b0f456e13bda44c5 as hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/591a7ecb679a4fe6b0f456e13bda44c5 2023-07-18 04:54:50,975 INFO [M:0;jenkins-hbase4:36243] regionserver.HStore(1080): Added hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/591a7ecb679a4fe6b0f456e13bda44c5, entries=10, sequenceid=217, filesize=5.8 K 2023-07-18 04:54:50,976 INFO [M:0;jenkins-hbase4:36243] regionserver.HRegion(2948): Finished flush of dataSize ~29.39 KB/30093, heapSize ~36.53 KB/37408, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 22ms, sequenceid=217, compaction requested=false 2023-07-18 04:54:50,978 INFO [M:0;jenkins-hbase4:36243] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:54:50,978 DEBUG [M:0;jenkins-hbase4:36243] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 04:54:50,978 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/MasterData/WALs/jenkins-hbase4.apache.org,36243,1689656083152 2023-07-18 04:54:50,984 INFO [M:0;jenkins-hbase4:36243] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-18 04:54:50,984 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 04:54:50,984 INFO [M:0;jenkins-hbase4:36243] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36243 2023-07-18 04:54:50,989 DEBUG [M:0;jenkins-hbase4:36243] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,36243,1689656083152 already deleted, retry=false 2023-07-18 04:54:51,047 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:45615-0x10176c7bf3c0029, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:51,047 INFO [RS:0;jenkins-hbase4:45615] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45615,1689656083190; zookeeper connection closed. 2023-07-18 04:54:51,047 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:45615-0x10176c7bf3c0029, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:51,048 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2dd1b216] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2dd1b216 2023-07-18 04:54:51,048 INFO [Listener at localhost/37011] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-18 04:54:51,146 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:51,147 INFO [M:0;jenkins-hbase4:36243] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36243,1689656083152; zookeeper connection closed. 2023-07-18 04:54:51,147 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:36243-0x10176c7bf3c0028, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:54:51,150 INFO [Listener at localhost/37011] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 04:54:51,150 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:51,151 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:51,151 INFO [Listener at localhost/37011] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 04:54:51,151 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:51,151 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 04:54:51,151 INFO [Listener at localhost/37011] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 04:54:51,153 INFO [Listener at localhost/37011] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41819 2023-07-18 04:54:51,153 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:51,155 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:51,156 INFO [Listener at localhost/37011] zookeeper.RecoverableZooKeeper(93): Process identifier=master:41819 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:51,159 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:418190x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:51,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:41819-0x10176c7bf3c0030 connected 2023-07-18 04:54:51,163 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 04:54:51,164 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:51,164 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 04:54:51,166 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41819 2023-07-18 04:54:51,167 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41819 2023-07-18 04:54:51,167 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41819 2023-07-18 04:54:51,167 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41819 2023-07-18 04:54:51,167 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41819 2023-07-18 04:54:51,168 INFO [Listener at localhost/37011] master.HMaster(444): hbase.rootdir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0, hbase.cluster.distributed=false 2023-07-18 04:54:51,171 DEBUG [pool-652-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-18 04:54:51,183 INFO [Listener at localhost/37011] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-18 04:54:51,183 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:51,183 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:51,183 INFO [Listener at localhost/37011] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-18 04:54:51,183 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-18 04:54:51,183 INFO [Listener at localhost/37011] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-18 04:54:51,183 INFO [Listener at localhost/37011] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-18 04:54:51,187 INFO [Listener at localhost/37011] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34513 2023-07-18 04:54:51,188 INFO [Listener at localhost/37011] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-18 04:54:51,189 DEBUG [Listener at localhost/37011] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-18 04:54:51,189 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:51,190 INFO [Listener at localhost/37011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:51,191 INFO [Listener at localhost/37011] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34513 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:51,193 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:345130x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:51,195 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:345130x0, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 04:54:51,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34513-0x10176c7bf3c0031 connected 2023-07-18 04:54:51,196 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:34513-0x10176c7bf3c0031, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:54:51,196 DEBUG [Listener at localhost/37011] zookeeper.ZKUtil(164): regionserver:34513-0x10176c7bf3c0031, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-18 04:54:51,198 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34513 2023-07-18 04:54:51,199 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34513 2023-07-18 04:54:51,199 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34513 2023-07-18 04:54:51,203 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34513 2023-07-18 04:54:51,203 DEBUG [Listener at localhost/37011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34513 2023-07-18 04:54:51,204 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,41819,1689656091150 2023-07-18 04:54:51,206 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-18 04:54:51,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,41819,1689656091150 2023-07-18 04:54:51,208 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 04:54:51,208 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:34513-0x10176c7bf3c0031, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-18 04:54:51,208 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:51,209 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-18 04:54:51,211 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,41819,1689656091150 from backup master directory 2023-07-18 04:54:51,211 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-18 04:54:51,213 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,41819,1689656091150 2023-07-18 04:54:51,213 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 04:54:51,213 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-18 04:54:51,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,41819,1689656091150 2023-07-18 04:54:51,225 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-18 04:54:51,247 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x0ece6086 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:51,251 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@286ccfe6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:51,251 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-18 04:54:51,252 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-18 04:54:51,252 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:51,252 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:51,258 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,33929,1689656066387 to hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,33929,1689656066387-dead as it is dead 2023-07-18 04:54:51,258 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,33929,1689656066387-dead/jenkins-hbase4.apache.org%2C33929%2C1689656066387.jenkins-hbase4.apache.org%2C33929%2C1689656066387.regiongroup-0.1689656066509 2023-07-18 04:54:51,259 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,33929,1689656066387-dead/jenkins-hbase4.apache.org%2C33929%2C1689656066387.jenkins-hbase4.apache.org%2C33929%2C1689656066387.regiongroup-0.1689656066509 after 1ms 2023-07-18 04:54:51,259 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,33929,1689656066387-dead/jenkins-hbase4.apache.org%2C33929%2C1689656066387.jenkins-hbase4.apache.org%2C33929%2C1689656066387.regiongroup-0.1689656066509 to hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33929%2C1689656066387.jenkins-hbase4.apache.org%2C33929%2C1689656066387.regiongroup-0.1689656066509 2023-07-18 04:54:51,259 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,33929,1689656066387-dead 2023-07-18 04:54:51,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,41819,1689656091150 2023-07-18 04:54:51,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:51,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41819%2C1689656091150.jenkins-hbase4.apache.org%2C41819%2C1689656091150.regiongroup-0, suffix=, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,41819,1689656091150, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/oldWALs, maxLogs=10 2023-07-18 04:54:51,277 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,41819,1689656091150/jenkins-hbase4.apache.org%2C41819%2C1689656091150.jenkins-hbase4.apache.org%2C41819%2C1689656091150.regiongroup-0.1689656091262 2023-07-18 04:54:51,277 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38949,DS-7e9427a5-d2d9-4ece-8da8-f6fb77b8864e,DISK], DatanodeInfoWithStorage[127.0.0.1:40889,DS-cb62bdfa-114c-4e1c-89f3-b40b9c3afb83,DISK], DatanodeInfoWithStorage[127.0.0.1:44317,DS-d0c3f250-f9ef-4d3a-8da0-904990569608,DISK]] 2023-07-18 04:54:51,277 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:51,278 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:51,278 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:51,278 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:51,279 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:51,280 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-18 04:54:51,280 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-18 04:54:51,285 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2bdbc267c2bb4e088935a47685bc6cbe 2023-07-18 04:54:51,289 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e1b8c52d3f0c47a3b53f87b774c972d5 2023-07-18 04:54:51,289 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:51,289 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-18 04:54:51,289 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33929%2C1689656066387.jenkins-hbase4.apache.org%2C33929%2C1689656066387.regiongroup-0.1689656066509 2023-07-18 04:54:51,293 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 115, firstSequenceIdInLog=120, maxSequenceIdInLog=219, path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33929%2C1689656066387.jenkins-hbase4.apache.org%2C33929%2C1689656066387.regiongroup-0.1689656066509 2023-07-18 04:54:51,293 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33929%2C1689656066387.jenkins-hbase4.apache.org%2C33929%2C1689656066387.regiongroup-0.1689656066509 2023-07-18 04:54:51,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-18 04:54:51,298 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/219.seqid, newMaxSeqId=219, maxSeqId=118 2023-07-18 04:54:51,299 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=220; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60984133, jitterRate=-0.09126560389995575}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 04:54:51,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 04:54:51,299 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-18 04:54:51,301 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-18 04:54:51,301 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-18 04:54:51,301 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-18 04:54:51,301 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-18 04:54:51,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-18 04:54:51,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-18 04:54:51,309 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-18 04:54:51,309 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-18 04:54:51,310 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-18 04:54:51,310 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-18 04:54:51,310 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,43065,1689656057946, splitWal=true, meta=false 2023-07-18 04:54:51,311 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,36167,1689656057977, splitWal=true, meta=false 2023-07-18 04:54:51,311 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,42145,1689656058011, splitWal=true, meta=true 2023-07-18 04:54:51,311 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,43893,1689656057910, splitWal=true, meta=false 2023-07-18 04:54:51,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-18 04:54:51,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-18 04:54:51,312 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-18 04:54:51,313 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37265,1689656066415, table=hbase:meta, region=1588230740 2023-07-18 04:54:51,314 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 1 possibly 'live' servers, and 0 'splitting'. 2023-07-18 04:54:51,319 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,37265,1689656066415 already deleted, retry=false 2023-07-18 04:54:51,319 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,37265,1689656066415 on jenkins-hbase4.apache.org,41819,1689656091150 2023-07-18 04:54:51,322 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,37265,1689656066415, splitWal=true, meta=true 2023-07-18 04:54:51,322 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=27 for jenkins-hbase4.apache.org,37265,1689656066415 (carryingMeta=true) jenkins-hbase4.apache.org,37265,1689656066415/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@e210c3e[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-18 04:54:51,335 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-18 04:54:51,336 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-18 04:54:51,336 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-18 04:54:51,336 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-18 04:54:51,337 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-18 04:54:51,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-18 04:54:51,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-18 04:54:51,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-18 04:54:51,340 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 04:54:51,340 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:34513-0x10176c7bf3c0031, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-18 04:54:51,340 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:54:51,340 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,41819,1689656091150, sessionid=0x10176c7bf3c0030, setting cluster-up flag (Was=false) 2023-07-18 04:54:51,345 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-18 04:54:51,346 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41819,1689656091150 2023-07-18 04:54:51,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-18 04:54:51,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41819,1689656091150 2023-07-18 04:54:51,349 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/.hbase-snapshot/.tmp 2023-07-18 04:54:51,350 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:51,350 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:51,350 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:51,350 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-18 04:54:51,350 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-18 04:54:51,351 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:51,351 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 04:54:51,351 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:51,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689656121352 2023-07-18 04:54:51,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-18 04:54:51,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-18 04:54:51,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-18 04:54:51,353 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-18 04:54:51,353 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-18 04:54:51,353 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-18 04:54:51,353 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:51,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-18 04:54:51,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-18 04:54:51,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-18 04:54:51,354 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37265,1689656066415; numProcessing=1 2023-07-18 04:54:51,355 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-18 04:54:51,355 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=27, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,37265,1689656066415, splitWal=true, meta=true 2023-07-18 04:54:51,355 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-18 04:54:51,356 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656091355,5,FailOnTimeoutGroup] 2023-07-18 04:54:51,356 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656091356,5,FailOnTimeoutGroup] 2023-07-18 04:54:51,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:51,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-18 04:54:51,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:51,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:51,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689656091356, completionTime=-1 2023-07-18 04:54:51,356 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-18 04:54:51,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-18 04:54:51,357 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,37265,1689656066415, splitWal=true, meta=true, isMeta: true 2023-07-18 04:54:51,358 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415-splitting 2023-07-18 04:54:51,359 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415-splitting dir is empty, no logs to split. 2023-07-18 04:54:51,359 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,37265,1689656066415 WAL count=0, meta=true 2023-07-18 04:54:51,361 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415-splitting dir is empty, no logs to split. 2023-07-18 04:54:51,361 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,37265,1689656066415 WAL count=0, meta=true 2023-07-18 04:54:51,362 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,37265,1689656066415 WAL splitting is done? wals=0, meta=true 2023-07-18 04:54:51,363 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-18 04:54:51,364 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-18 04:54:51,366 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 04:54:51,404 INFO [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(951): ClusterId : b492e8cd-f4f9-4591-97c6-7f2edd4114a8 2023-07-18 04:54:51,404 DEBUG [RS:0;jenkins-hbase4:34513] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-18 04:54:51,407 DEBUG [RS:0;jenkins-hbase4:34513] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-18 04:54:51,407 DEBUG [RS:0;jenkins-hbase4:34513] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-18 04:54:51,409 DEBUG [RS:0;jenkins-hbase4:34513] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-18 04:54:51,411 DEBUG [RS:0;jenkins-hbase4:34513] zookeeper.ReadOnlyZKClient(139): Connect 0x39f6081c to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:51,418 DEBUG [RS:0;jenkins-hbase4:34513] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3731f152, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:51,418 DEBUG [RS:0;jenkins-hbase4:34513] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@330e636c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 04:54:51,427 DEBUG [RS:0;jenkins-hbase4:34513] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:34513 2023-07-18 04:54:51,427 INFO [RS:0;jenkins-hbase4:34513] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-18 04:54:51,427 INFO [RS:0;jenkins-hbase4:34513] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-18 04:54:51,427 DEBUG [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(1022): About to register with Master. 2023-07-18 04:54:51,427 INFO [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41819,1689656091150 with isa=jenkins-hbase4.apache.org/172.31.14.131:34513, startcode=1689656091182 2023-07-18 04:54:51,428 DEBUG [RS:0;jenkins-hbase4:34513] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-18 04:54:51,431 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43057, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=RegionServerStatusService 2023-07-18 04:54:51,432 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41819] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:54:51,433 DEBUG [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0 2023-07-18 04:54:51,433 DEBUG [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37939 2023-07-18 04:54:51,433 DEBUG [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-18 04:54:51,434 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:51,434 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10176c7bf3c0026, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:54:51,434 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 04:54:51,435 DEBUG [RS:0;jenkins-hbase4:34513] zookeeper.ZKUtil(162): regionserver:34513-0x10176c7bf3c0031, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:54:51,436 WARN [RS:0;jenkins-hbase4:34513] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-18 04:54:51,436 INFO [RS:0;jenkins-hbase4:34513] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:51,436 INFO [RS:0;jenkins-hbase4:34513] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:51,436 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34513,1689656091182] 2023-07-18 04:54:51,436 DEBUG [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:54:51,440 DEBUG [RS:0;jenkins-hbase4:34513] zookeeper.ZKUtil(162): regionserver:34513-0x10176c7bf3c0031, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:54:51,441 DEBUG [RS:0;jenkins-hbase4:34513] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 04:54:51,441 INFO [RS:0;jenkins-hbase4:34513] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-18 04:54:51,442 INFO [RS:0;jenkins-hbase4:34513] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-18 04:54:51,443 INFO [RS:0;jenkins-hbase4:34513] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-18 04:54:51,443 INFO [RS:0;jenkins-hbase4:34513] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:51,443 INFO [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-18 04:54:51,445 INFO [RS:0;jenkins-hbase4:34513] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:51,446 DEBUG [RS:0;jenkins-hbase4:34513] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:51,446 DEBUG [RS:0;jenkins-hbase4:34513] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:51,446 DEBUG [RS:0;jenkins-hbase4:34513] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:51,446 DEBUG [RS:0;jenkins-hbase4:34513] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:51,446 DEBUG [RS:0;jenkins-hbase4:34513] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:51,446 DEBUG [RS:0;jenkins-hbase4:34513] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-18 04:54:51,446 DEBUG [RS:0;jenkins-hbase4:34513] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:51,446 DEBUG [RS:0;jenkins-hbase4:34513] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:51,447 DEBUG [RS:0;jenkins-hbase4:34513] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:51,447 DEBUG [RS:0;jenkins-hbase4:34513] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-18 04:54:51,454 INFO [RS:0;jenkins-hbase4:34513] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:51,454 INFO [RS:0;jenkins-hbase4:34513] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:51,454 INFO [RS:0;jenkins-hbase4:34513] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:51,457 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=100ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-18 04:54:51,473 INFO [RS:0;jenkins-hbase4:34513] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-18 04:54:51,474 INFO [RS:0;jenkins-hbase4:34513] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34513,1689656091182-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:51,489 INFO [RS:0;jenkins-hbase4:34513] regionserver.Replication(203): jenkins-hbase4.apache.org,34513,1689656091182 started 2023-07-18 04:54:51,489 INFO [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34513,1689656091182, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34513, sessionid=0x10176c7bf3c0031 2023-07-18 04:54:51,489 DEBUG [RS:0;jenkins-hbase4:34513] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-18 04:54:51,489 DEBUG [RS:0;jenkins-hbase4:34513] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:54:51,489 DEBUG [RS:0;jenkins-hbase4:34513] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34513,1689656091182' 2023-07-18 04:54:51,489 DEBUG [RS:0;jenkins-hbase4:34513] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-18 04:54:51,490 DEBUG [RS:0;jenkins-hbase4:34513] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-18 04:54:51,490 DEBUG [RS:0;jenkins-hbase4:34513] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-18 04:54:51,490 DEBUG [RS:0;jenkins-hbase4:34513] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-18 04:54:51,490 DEBUG [RS:0;jenkins-hbase4:34513] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:54:51,490 DEBUG [RS:0;jenkins-hbase4:34513] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34513,1689656091182' 2023-07-18 04:54:51,490 DEBUG [RS:0;jenkins-hbase4:34513] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-18 04:54:51,491 DEBUG [RS:0;jenkins-hbase4:34513] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-18 04:54:51,491 DEBUG [RS:0;jenkins-hbase4:34513] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-18 04:54:51,491 INFO [RS:0;jenkins-hbase4:34513] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-18 04:54:51,491 INFO [RS:0;jenkins-hbase4:34513] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-18 04:54:51,516 DEBUG [jenkins-hbase4:41819] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-18 04:54:51,517 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34513,1689656091182, state=OPENING 2023-07-18 04:54:51,520 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-18 04:54:51,520 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-18 04:54:51,520 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34513,1689656091182}] 2023-07-18 04:54:51,591 INFO [RS:0;jenkins-hbase4:34513] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:51,593 INFO [RS:0;jenkins-hbase4:34513] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34513%2C1689656091182.jenkins-hbase4.apache.org%2C34513%2C1689656091182.regiongroup-0, suffix=, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,34513,1689656091182, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs, maxLogs=10 2023-07-18 04:54:51,603 INFO [RS:0;jenkins-hbase4:34513] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,34513,1689656091182/jenkins-hbase4.apache.org%2C34513%2C1689656091182.jenkins-hbase4.apache.org%2C34513%2C1689656091182.regiongroup-0.1689656091594 2023-07-18 04:54:51,603 DEBUG [RS:0;jenkins-hbase4:34513] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37603,DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347,DISK], DatanodeInfoWithStorage[127.0.0.1:44317,DS-d0c3f250-f9ef-4d3a-8da0-904990569608,DISK], DatanodeInfoWithStorage[127.0.0.1:40889,DS-cb62bdfa-114c-4e1c-89f3-b40b9c3afb83,DISK]] 2023-07-18 04:54:51,677 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:54:51,677 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 04:54:51,680 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44266, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 04:54:51,684 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-18 04:54:51,684 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:51,684 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:51,684 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:51,685 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34513%2C1689656091182.meta, suffix=.meta, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,34513,1689656091182, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs, maxLogs=10 2023-07-18 04:54:51,695 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,34513,1689656091182/jenkins-hbase4.apache.org%2C34513%2C1689656091182.meta.1689656091686.meta 2023-07-18 04:54:51,695 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44317,DS-d0c3f250-f9ef-4d3a-8da0-904990569608,DISK], DatanodeInfoWithStorage[127.0.0.1:38949,DS-7e9427a5-d2d9-4ece-8da8-f6fb77b8864e,DISK], DatanodeInfoWithStorage[127.0.0.1:37603,DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347,DISK]] 2023-07-18 04:54:51,695 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:51,695 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-18 04:54:51,695 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-18 04:54:51,695 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-18 04:54:51,695 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-18 04:54:51,695 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:51,695 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-18 04:54:51,695 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-18 04:54:51,697 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-18 04:54:51,698 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info 2023-07-18 04:54:51,698 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info 2023-07-18 04:54:51,698 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-18 04:54:51,704 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info/2e79da83316e4b4cb208a78c9bd67186 2023-07-18 04:54:51,707 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info/d057a108d301417792b3fe3bc3fbdfc0 2023-07-18 04:54:51,707 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:51,707 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-18 04:54:51,708 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/rep_barrier 2023-07-18 04:54:51,708 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/rep_barrier 2023-07-18 04:54:51,709 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-18 04:54:51,709 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:51,709 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-18 04:54:51,710 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/table 2023-07-18 04:54:51,710 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/table 2023-07-18 04:54:51,710 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-18 04:54:51,717 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/table/bd05a84742aa4c3d88ab2415d20b934f 2023-07-18 04:54:51,717 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:51,718 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740 2023-07-18 04:54:51,719 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740 2023-07-18 04:54:51,721 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-18 04:54:51,723 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-18 04:54:51,723 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=35; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58737562, jitterRate=-0.12474212050437927}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-18 04:54:51,724 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-18 04:54:51,726 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=29, masterSystemTime=1689656091677 2023-07-18 04:54:51,730 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-18 04:54:51,731 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-18 04:54:51,731 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34513,1689656091182, state=OPEN 2023-07-18 04:54:51,733 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-18 04:54:51,733 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-18 04:54:51,736 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-07-18 04:54:51,736 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34513,1689656091182 in 213 msec 2023-07-18 04:54:51,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-07-18 04:54:51,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 373 msec 2023-07-18 04:54:52,959 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1602ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-07-18 04:54:53,780 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 04:54:54,462 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3105ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-18 04:54:55,864 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-18 04:54:55,864 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-18 04:54:55,868 DEBUG [hconnection-0x3834e36e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:55,870 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44278, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:55,878 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=383bb30520997d1bb825dc5e5f31999e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37265,1689656066415, regionLocation=jenkins-hbase4.apache.org,37265,1689656066415, openSeqNum=10 2023-07-18 04:54:55,878 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=a2e71ceab1ce1a583f43e863ff7cd0a0, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37265,1689656066415, regionLocation=jenkins-hbase4.apache.org,37265,1689656066415, openSeqNum=11 2023-07-18 04:54:55,878 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=1b9326fa16ae8497bb1df12a88dfd577, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37265,1689656066415, regionLocation=jenkins-hbase4.apache.org,37265,1689656066415, openSeqNum=12 2023-07-18 04:54:55,878 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-18 04:54:55,878 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689656155878 2023-07-18 04:54:55,878 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689656215878 2023-07-18 04:54:55,878 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 14 msec 2023-07-18 04:54:55,895 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,37265,1689656066415 had 4 regions 2023-07-18 04:54:55,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41819,1689656091150-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:55,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41819,1689656091150-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:55,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41819,1689656091150-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:55,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:41819, period=300000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:55,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-18 04:54:55,896 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. is NOT online; state={383bb30520997d1bb825dc5e5f31999e state=OPEN, ts=1689656095878, server=jenkins-hbase4.apache.org,37265,1689656066415}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-18 04:54:55,897 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,37265,1689656066415, splitWal=true, meta=true, isMeta: false 2023-07-18 04:54:55,900 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415-splitting dir is empty, no logs to split. 2023-07-18 04:54:55,900 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,37265,1689656066415 WAL count=0, meta=false 2023-07-18 04:54:55,900 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,37265,1689656066415/hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e., unknown_server=jenkins-hbase4.apache.org,37265,1689656066415/t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0., unknown_server=jenkins-hbase4.apache.org,37265,1689656066415/t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:55,902 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,37265,1689656066415-splitting dir is empty, no logs to split. 2023-07-18 04:54:55,902 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,37265,1689656066415 WAL count=0, meta=false 2023-07-18 04:54:55,902 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,37265,1689656066415 WAL splitting is done? wals=0, meta=false 2023-07-18 04:54:55,906 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=383bb30520997d1bb825dc5e5f31999e, ASSIGN}, {pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=1b9326fa16ae8497bb1df12a88dfd577, ASSIGN}, {pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=a2e71ceab1ce1a583f43e863ff7cd0a0, ASSIGN}] 2023-07-18 04:54:55,907 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=a2e71ceab1ce1a583f43e863ff7cd0a0, ASSIGN 2023-07-18 04:54:55,907 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=383bb30520997d1bb825dc5e5f31999e, ASSIGN 2023-07-18 04:54:55,907 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=1b9326fa16ae8497bb1df12a88dfd577, ASSIGN 2023-07-18 04:54:55,908 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=a2e71ceab1ce1a583f43e863ff7cd0a0, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 04:54:55,908 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=1b9326fa16ae8497bb1df12a88dfd577, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 04:54:55,908 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=383bb30520997d1bb825dc5e5f31999e, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-18 04:54:55,909 DEBUG [jenkins-hbase4:41819] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-18 04:54:55,912 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=a2e71ceab1ce1a583f43e863ff7cd0a0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:54:55,912 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=1b9326fa16ae8497bb1df12a88dfd577, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:54:55,913 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656095912"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656095912"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656095912"}]},"ts":"1689656095912"} 2023-07-18 04:54:55,913 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656095912"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656095912"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656095912"}]},"ts":"1689656095912"} 2023-07-18 04:54:55,913 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=383bb30520997d1bb825dc5e5f31999e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:54:55,913 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689656095913"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689656095913"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689656095913"}]},"ts":"1689656095913"} 2023-07-18 04:54:55,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure a2e71ceab1ce1a583f43e863ff7cd0a0, server=jenkins-hbase4.apache.org,34513,1689656091182}] 2023-07-18 04:54:55,916 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure 1b9326fa16ae8497bb1df12a88dfd577, server=jenkins-hbase4.apache.org,34513,1689656091182}] 2023-07-18 04:54:55,919 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=35, ppid=30, state=RUNNABLE; OpenRegionProcedure 383bb30520997d1bb825dc5e5f31999e, server=jenkins-hbase4.apache.org,34513,1689656091182}] 2023-07-18 04:54:56,073 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:56,073 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-18 04:54:56,074 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34513%2C1689656091182.jenkins-hbase4.apache.org%2C34513%2C1689656091182.regiongroup-1, suffix=, logDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,34513,1689656091182, archiveDir=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs, maxLogs=10 2023-07-18 04:54:56,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:56,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a2e71ceab1ce1a583f43e863ff7cd0a0, NAME => 't1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:56,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:56,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:56,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:56,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:56,077 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:56,078 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1 2023-07-18 04:54:56,078 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1 2023-07-18 04:54:56,079 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2e71ceab1ce1a583f43e863ff7cd0a0 columnFamilyName cf1 2023-07-18 04:54:56,085 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,34513,1689656091182/jenkins-hbase4.apache.org%2C34513%2C1689656091182.jenkins-hbase4.apache.org%2C34513%2C1689656091182.regiongroup-1.1689656096075 2023-07-18 04:54:56,085 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40889,DS-cb62bdfa-114c-4e1c-89f3-b40b9c3afb83,DISK], DatanodeInfoWithStorage[127.0.0.1:37603,DS-f5b7f7b4-3d03-4dc7-8aac-3a5a15939347,DISK], DatanodeInfoWithStorage[127.0.0.1:44317,DS-d0c3f250-f9ef-4d3a-8da0-904990569608,DISK]] 2023-07-18 04:54:56,085 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 383bb30520997d1bb825dc5e5f31999e, NAME => 'hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:56,085 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:56,085 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:56,086 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:56,086 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:56,087 INFO [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:56,089 DEBUG [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/info 2023-07-18 04:54:56,089 DEBUG [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/info 2023-07-18 04:54:56,089 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1/cc4f63e41e55476a976035dc43508488 2023-07-18 04:54:56,089 INFO [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 383bb30520997d1bb825dc5e5f31999e columnFamilyName info 2023-07-18 04:54:56,093 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d6a832002bf544daad6d8b1c1648d003 2023-07-18 04:54:56,094 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1/d6a832002bf544daad6d8b1c1648d003 2023-07-18 04:54:56,094 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(310): Store=a2e71ceab1ce1a583f43e863ff7cd0a0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:56,094 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:56,095 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/norep 2023-07-18 04:54:56,095 DEBUG [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/norep 2023-07-18 04:54:56,095 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2e71ceab1ce1a583f43e863ff7cd0a0 columnFamilyName norep 2023-07-18 04:54:56,095 DEBUG [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/info/fad48ef599d54b4b956ee92103e35a9a 2023-07-18 04:54:56,096 INFO [StoreOpener-383bb30520997d1bb825dc5e5f31999e-1] regionserver.HStore(310): Store=383bb30520997d1bb825dc5e5f31999e/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:56,096 INFO [StoreOpener-a2e71ceab1ce1a583f43e863ff7cd0a0-1] regionserver.HStore(310): Store=a2e71ceab1ce1a583f43e863ff7cd0a0/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:56,096 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:56,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:56,098 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:56,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:56,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 04:54:56,101 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:54:56,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:54:56,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a2e71ceab1ce1a583f43e863ff7cd0a0; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73133841, jitterRate=0.08977915346622467}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 04:54:56,102 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 383bb30520997d1bb825dc5e5f31999e; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61084150, jitterRate=-0.08977523446083069}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-18 04:54:56,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a2e71ceab1ce1a583f43e863ff7cd0a0: 2023-07-18 04:54:56,102 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 383bb30520997d1bb825dc5e5f31999e: 2023-07-18 04:54:56,105 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e., pid=35, masterSystemTime=1689656096069 2023-07-18 04:54:56,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0., pid=33, masterSystemTime=1689656096069 2023-07-18 04:54:56,107 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:56,107 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:54:56,108 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=383bb30520997d1bb825dc5e5f31999e, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:54:56,108 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689656096108"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656096108"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656096108"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656096108"}]},"ts":"1689656096108"} 2023-07-18 04:54:56,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:56,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:54:56,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:56,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1b9326fa16ae8497bb1df12a88dfd577, NAME => 't2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.', STARTKEY => '', ENDKEY => ''} 2023-07-18 04:54:56,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:56,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-18 04:54:56,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:56,109 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=a2e71ceab1ce1a583f43e863ff7cd0a0, regionState=OPEN, openSeqNum=18, regionLocation=jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:54:56,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:56,109 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656096109"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656096109"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656096109"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656096109"}]},"ts":"1689656096109"} 2023-07-18 04:54:56,112 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:56,112 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=35, resume processing ppid=30 2023-07-18 04:54:56,113 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1 2023-07-18 04:54:56,113 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1 2023-07-18 04:54:56,113 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=30, state=SUCCESS; OpenRegionProcedure 383bb30520997d1bb825dc5e5f31999e, server=jenkins-hbase4.apache.org,34513,1689656091182 in 193 msec 2023-07-18 04:54:56,113 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b9326fa16ae8497bb1df12a88dfd577 columnFamilyName cf1 2023-07-18 04:54:56,113 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-07-18 04:54:56,113 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure a2e71ceab1ce1a583f43e863ff7cd0a0, server=jenkins-hbase4.apache.org,34513,1689656091182 in 196 msec 2023-07-18 04:54:56,114 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=383bb30520997d1bb825dc5e5f31999e, ASSIGN in 209 msec 2023-07-18 04:54:56,115 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=a2e71ceab1ce1a583f43e863ff7cd0a0, ASSIGN in 209 msec 2023-07-18 04:54:56,120 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1/bbbbe1a481884a64ac20a4d86fe979a7 2023-07-18 04:54:56,128 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e685f233fda344e68e26feddcadc4b60 2023-07-18 04:54:56,128 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(539): loaded hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1/e685f233fda344e68e26feddcadc4b60 2023-07-18 04:54:56,128 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(310): Store=1b9326fa16ae8497bb1df12a88dfd577/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:56,128 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:56,129 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/norep 2023-07-18 04:54:56,129 DEBUG [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/norep 2023-07-18 04:54:56,129 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b9326fa16ae8497bb1df12a88dfd577 columnFamilyName norep 2023-07-18 04:54:56,130 INFO [StoreOpener-1b9326fa16ae8497bb1df12a88dfd577-1] regionserver.HStore(310): Store=1b9326fa16ae8497bb1df12a88dfd577/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-18 04:54:56,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:56,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:56,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-18 04:54:56,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:54:56,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1b9326fa16ae8497bb1df12a88dfd577; next sequenceid=19; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62944221, jitterRate=-0.062058016657829285}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-18 04:54:56,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1b9326fa16ae8497bb1df12a88dfd577: 2023-07-18 04:54:56,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577., pid=34, masterSystemTime=1689656096069 2023-07-18 04:54:56,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:56,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:54:56,138 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=1b9326fa16ae8497bb1df12a88dfd577, regionState=OPEN, openSeqNum=19, regionLocation=jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:54:56,138 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689656096138"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689656096138"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689656096138"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689656096138"}]},"ts":"1689656096138"} 2023-07-18 04:54:56,142 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-07-18 04:54:56,142 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure 1b9326fa16ae8497bb1df12a88dfd577, server=jenkins-hbase4.apache.org,34513,1689656091182 in 224 msec 2023-07-18 04:54:56,144 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=27 2023-07-18 04:54:56,144 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,37265,1689656066415 after splitting done 2023-07-18 04:54:56,144 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=1b9326fa16ae8497bb1df12a88dfd577, ASSIGN in 238 msec 2023-07-18 04:54:56,144 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,37265,1689656066415 from processing; numProcessing=0 2023-07-18 04:54:56,146 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,37265,1689656066415, splitWal=true, meta=true in 4.8250 sec 2023-07-18 04:54:56,489 INFO [jenkins-hbase4:34513Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 04:54:56,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-18 04:54:56,911 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-18 04:54:56,914 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-18 04:54:56,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.700sec 2023-07-18 04:54:56,915 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-18 04:54:56,915 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-18 04:54:56,915 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-18 04:54:56,915 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41819,1689656091150-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-18 04:54:56,915 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41819,1689656091150-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-18 04:54:56,918 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-18 04:54:57,008 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(139): Connect 0x386733f1 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:57,013 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2dda6aed, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:57,014 DEBUG [hconnection-0x48a479f1-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:57,016 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44292, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:57,018 INFO [Listener at localhost/37011] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-18 04:54:57,018 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x386733f1 to 127.0.0.1:63236 2023-07-18 04:54:57,018 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:54:57,019 INFO [Listener at localhost/37011] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:41819 after: jenkins-hbase4.apache.org:41819 2023-07-18 04:54:57,020 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(139): Connect 0x52904104 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:57,024 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3990c156, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:57,441 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-18 04:54:57,442 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-18 04:54:57,442 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-18 04:54:57,442 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-18 04:54:57,526 DEBUG [hconnection-0x401fdd87-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:54:57,531 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44300, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-18 04:54:57,554 INFO [Listener at localhost/37011] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689656097554 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:57,559 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): syncupReplication16896560975540x0, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:57,560 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689656097554-0x10176c7bf3c0036 connected Start Replication Server start 2023-07-18 04:54:57,562 INFO [Listener at localhost/37011] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-18 04:54:57,562 INFO [Listener at localhost/37011] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-18 04:54:57,574 INFO [Listener at localhost/37011] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-18 04:54:57,575 DEBUG [Listener at localhost/37011] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-18 04:54:57,585 INFO [Listener at localhost/37011] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-18 04:54:57,587 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,45615,1689656083190] other RSs: [] 2023-07-18 04:54:57,605 DEBUG [Listener at localhost/37011.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x08186706 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:54:57,610 DEBUG [Listener at localhost/37011.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@45e4da90, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:54:57,610 INFO [Listener at localhost/37011.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:54:57,612 DEBUG [Listener at localhost/37011.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:54:57,613 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10176c7bf3c0038 connected 2023-07-18 04:54:57,614 INFO [Listener at localhost/37011.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=4b2ea01c-df7d-4e18-8185-4db3e50f17db to cluster=b492e8cd-f4f9-4591-97c6-7f2edd4114a8 2023-07-18 04:55:00,484 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,45615,1689656083190/1's WALs to 1689656097562.syncuptool.replication.org,1234,1 2023-07-18 04:55:00,487 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0.1689656083619 with data PBUF\x08\xC4\x03 2023-07-18 04:55:00,488 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1.1689656088273 with data PBUF\x08\x99\x06 2023-07-18 04:55:00,489 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,45615,1689656083190/1's WALs to 1689656097562.syncuptool.replication.org,1234,1 2023-07-18 04:55:01,489 INFO [jenkins-hbase4:34513Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 04:55:01,579 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 04:55:02,494 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,45615,1689656083190] 2023-07-18 04:55:02,494 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,45615,1689656083190] 2023-07-18 04:55:02,505 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,45615,1689656083190] 2023-07-18 04:55:02,505 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,45615,1689656083190, ReplicationSource: 1, currentBandwidth=0 2023-07-18 04:55:02,505 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:55:02,525 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190] zookeeper.ReadOnlyZKClient(139): Connect 0x272235ca to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:55:02,531 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@696977ec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:55:02,531 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:63236 2023-07-18 04:55:02,533 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-18 04:55:02,535 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10176c7bf3c003a connected 2023-07-18 04:55:02,535 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,45615,1689656083190 (queues=2) is replicating from cluster=4b2ea01c-df7d-4e18-8185-4db3e50f17db to cluster=b492e8cd-f4f9-4591-97c6-7f2edd4114a8 2023-07-18 04:55:02,535 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0 2023-07-18 04:55:02,537 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,45615,1689656083190, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 04:55:02,538 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1 2023-07-18 04:55:02,538 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0 2023-07-18 04:55:02,539 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,45615,1689656083190, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-18 04:55:02,540 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1 2023-07-18 04:55:02,548 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs/jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0.1689656083619 2023-07-18 04:55:02,549 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceWALReader(162): Read 100 WAL entries eligible for replication 2023-07-18 04:55:02,549 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-18 04:55:02,550 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,45615,1689656083190 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/db11400df50c3fb2fe7bab3791162e0a/112=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/113=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/114=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/115=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/116=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/117=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/118=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/119=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/120=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/121=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/122=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/123=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/124=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/125=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/126=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/127=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/128=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/129=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/130=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/131=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/132=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/133=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/134=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/135=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/136=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/137=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/138=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/139=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/140=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/141=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/142=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/143=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/144=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/145=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/146=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/147=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/148=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/149=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/150=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/151=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/152=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/153=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/154=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/155=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/156=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/157=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/158=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/159=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/160=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/161=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/162=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/163=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/164=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/165=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/166=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/167=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/168=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/169=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/170=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/171=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/172=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/173=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/174=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/175=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/176=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/177=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/178=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/179=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/180=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/181=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/182=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/183=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/184=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/185=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/186=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/187=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/188=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/189=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/190=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/191=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/192=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/193=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/194=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/195=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/196=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/197=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/198=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/199=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/200=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/201=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/202=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/203=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/204=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/205=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/206=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/207=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/208=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/209=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/210=[#edits: 1 = ],208}, {t1_syncup/db11400df50c3fb2fe7bab3791162e0a/211=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs/jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0.1689656083619, lastWalPosition=11986, nbRowKeys=100, nbHFiles=0, heapSize=20800, lastSeqIds={}, endOfFile=true,usedBufferSize=20800] 2023-07-18 04:55:02,552 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs/jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1.1689656088273 2023-07-18 04:55:02,552 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceWALReader(162): Read 200 WAL entries eligible for replication 2023-07-18 04:55:02,553 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-18 04:55:02,554 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-18 04:55:02,555 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,45615,1689656083190 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/212=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/213=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/214=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/215=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/216=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/217=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/218=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/219=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/220=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/221=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/222=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/223=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/224=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/225=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/226=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/227=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/228=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/229=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/230=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/231=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/232=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/233=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/234=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/235=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/236=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/237=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/238=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/239=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/240=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/241=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/242=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/243=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/244=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/245=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/246=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/247=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/248=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/249=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/250=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/251=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/252=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/253=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/254=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/255=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/256=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/257=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/258=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/259=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/260=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/261=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/262=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/263=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/264=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/265=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/266=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/267=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/268=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/269=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/270=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/271=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/272=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/273=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/274=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/275=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/276=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/277=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/278=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/279=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/280=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/281=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/282=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/283=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/284=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/285=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/286=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/287=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/288=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/289=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/290=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/291=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/292=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/293=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/294=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/295=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/296=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/297=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/298=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/299=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/300=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/301=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/302=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/303=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/304=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/305=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/306=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/307=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/308=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/309=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/310=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/311=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/312=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/313=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/314=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/315=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/316=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/317=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/318=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/319=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/320=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/321=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/322=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/323=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/324=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/325=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/326=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/327=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/328=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/329=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/330=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/331=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/332=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/333=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/334=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/335=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/336=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/337=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/338=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/339=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/340=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/341=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/342=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/343=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/344=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/345=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/346=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/347=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/348=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/349=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/350=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/351=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/352=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/353=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/354=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/355=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/356=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/357=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/358=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/359=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/360=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/361=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/362=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/363=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/364=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/365=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/366=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/367=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/368=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/369=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/370=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/371=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/372=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/373=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/374=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/375=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/376=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/377=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/378=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/379=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/380=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/381=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/382=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/383=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/384=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/385=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/386=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/387=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/388=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/389=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/390=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/391=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/392=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/393=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/394=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/395=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/396=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/397=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/398=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/399=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/400=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/401=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/402=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/403=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/404=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/405=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/406=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/407=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/408=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/409=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/410=[#edits: 1 = ],208}, {t2_syncup/7f6a4e55b8779eead03ae75b46b98c60/411=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:32937/user/jenkins/test-data/59da311a-1137-4a18-5854-adffff7cba9c/oldWALs/jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1.1689656088273, lastWalPosition=23449, nbRowKeys=200, nbHFiles=0, heapSize=41600, lastSeqIds={}, endOfFile=true,usedBufferSize=41600] 2023-07-18 04:55:02,558 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53104, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-18 04:55:02,561 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34513] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 04:55:02,562 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34513] zookeeper.ReadOnlyZKClient(139): Connect 0x447a1cd0 to 127.0.0.1:63236 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-18 04:55:02,564 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=34513] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-18 04:55:02,565 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34513] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@4abdfa2f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-18 04:55:02,568 DEBUG [hconnection-0x4e8277a0-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-18 04:55:02,571 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53108, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=ClientService 2023-07-18 04:55:02,586 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34513] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 04:55:02,587 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1-jenkins-hbase4.apache.org,45615,1689656083190] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,45615,1689656083190] 2023-07-18 04:55:02,589 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0.1689656083619] 2023-07-18 04:55:02,591 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,45615,1689656083190 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-18 04:55:02,591 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:55:02,592 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=34513] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-18 04:55:02,593 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1-jenkins-hbase4.apache.org,45615,1689656083190] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,45615,1689656083190] 2023-07-18 04:55:02,594 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1.1689656088273] 2023-07-18 04:55:02,596 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,45615,1689656083190 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-18 04:55:02,596 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:55:02,692 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:55:02,694 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689656097554-0x10176c7bf3c0036, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689656097562.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:55:02,694 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-0,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,45615,1689656083190 with the following stats: Total replicated edits: 300, current progress: 2023-07-18 04:55:02,794 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,45615,1689656083190 2023-07-18 04:55:02,795 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1-jenkins-hbase4.apache.org,45615,1689656083190] zookeeper.ZKUtil(243): syncupReplication1689656097554-0x10176c7bf3c0036, quorum=127.0.0.1:63236, baseZNode=/1 Unable to list children of znode /1/replication/rs/1689656097562.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,45615,1689656083190 because node does not exist (not an error) 2023-07-18 04:55:02,795 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190.replicationSource.shipperjenkins-hbase4.apache.org%2C45615%2C1689656083190.jenkins-hbase4.apache.org%2C45615%2C1689656083190.regiongroup-1,1-jenkins-hbase4.apache.org,45615,1689656083190] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,45615,1689656083190 with the following stats: Total replicated edits: 300, current progress: 2023-07-18 04:55:06,489 INFO [jenkins-hbase4:34513Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-18 04:55:06,489 INFO [jenkins-hbase4:34513Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 300 2023-07-18 04:55:07,587 INFO [Listener at localhost/37011] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-18 04:55:07,693 DEBUG [Listener at localhost/37011.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10176c7bf3c0038, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:55:07,693 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x08186706 to 127.0.0.1:63236 2023-07-18 04:55:07,694 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:55:07,796 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689656097554-0x10176c7bf3c0036, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:55:07,796 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689656097554-0x10176c7bf3c0036, quorum=127.0.0.1:63236, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:55:07,805 INFO [Listener at localhost/37011] replication.TestReplicationSyncUpTool(280): SyncUpAfterPut succeeded at retry = 0 2023-07-18 04:55:07,805 INFO [Listener at localhost/37011] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-18 04:55:07,806 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x52904104 to 127.0.0.1:63236 2023-07-18 04:55:07,806 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:55:07,806 DEBUG [Listener at localhost/37011] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-18 04:55:07,806 DEBUG [Listener at localhost/37011] util.JVMClusterUtil(257): Found active master hash=1235855183, stopped=false 2023-07-18 04:55:07,806 INFO [Listener at localhost/37011] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,41819,1689656091150 2023-07-18 04:55:07,808 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:34513-0x10176c7bf3c0031, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 04:55:07,808 INFO [Listener at localhost/37011] procedure2.ProcedureExecutor(629): Stopping 2023-07-18 04:55:07,808 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-18 04:55:07,810 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:55:07,810 DEBUG [Listener at localhost/37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0ece6086 to 127.0.0.1:63236 2023-07-18 04:55:07,811 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34513-0x10176c7bf3c0031, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:55:07,811 DEBUG [Listener at localhost/37011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:55:07,811 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-18 04:55:07,811 INFO [Listener at localhost/37011] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34513,1689656091182' ***** 2023-07-18 04:55:07,812 INFO [Listener at localhost/37011] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-18 04:55:07,812 INFO [RS:0;jenkins-hbase4:34513] regionserver.HeapMemoryManager(220): Stopping 2023-07-18 04:55:07,812 INFO [RS:0;jenkins-hbase4:34513] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-18 04:55:07,812 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-18 04:55:07,812 INFO [RS:0;jenkins-hbase4:34513] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-18 04:55:07,813 INFO [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(3305): Received CLOSE for a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:55:07,814 INFO [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(3305): Received CLOSE for 1b9326fa16ae8497bb1df12a88dfd577 2023-07-18 04:55:07,814 INFO [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(3305): Received CLOSE for 383bb30520997d1bb825dc5e5f31999e 2023-07-18 04:55:07,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a2e71ceab1ce1a583f43e863ff7cd0a0, disabling compactions & flushes 2023-07-18 04:55:07,815 INFO [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:55:07,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:55:07,815 DEBUG [RS:0;jenkins-hbase4:34513] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x39f6081c to 127.0.0.1:63236 2023-07-18 04:55:07,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:55:07,815 DEBUG [RS:0;jenkins-hbase4:34513] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:55:07,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. after waiting 0 ms 2023-07-18 04:55:07,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:55:07,815 INFO [RS:0;jenkins-hbase4:34513] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-18 04:55:07,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing a2e71ceab1ce1a583f43e863ff7cd0a0 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-18 04:55:07,815 INFO [RS:0;jenkins-hbase4:34513] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-18 04:55:07,815 INFO [RS:0;jenkins-hbase4:34513] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-18 04:55:07,815 INFO [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-18 04:55:07,828 INFO [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-18 04:55:07,829 DEBUG [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(1478): Online Regions={a2e71ceab1ce1a583f43e863ff7cd0a0=t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0., 1588230740=hbase:meta,,1.1588230740, 1b9326fa16ae8497bb1df12a88dfd577=t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577., 383bb30520997d1bb825dc5e5f31999e=hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e.} 2023-07-18 04:55:07,829 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-18 04:55:07,829 DEBUG [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(1504): Waiting on 1588230740, 1b9326fa16ae8497bb1df12a88dfd577, 383bb30520997d1bb825dc5e5f31999e, a2e71ceab1ce1a583f43e863ff7cd0a0 2023-07-18 04:55:07,830 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-18 04:55:07,830 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-18 04:55:07,830 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-18 04:55:07,830 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-18 04:55:07,830 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-18 04:55:07,863 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=43 (bloomFilter=false), to=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/.tmp/info/ef71435e7c674e028950867c5770cbd2 2023-07-18 04:55:07,864 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=21 (bloomFilter=true), to=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/.tmp/cf1/ba881bf138a5441cb5b93140d550ada5 2023-07-18 04:55:07,869 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:55:07,871 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/.tmp/info/ef71435e7c674e028950867c5770cbd2 as hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info/ef71435e7c674e028950867c5770cbd2 2023-07-18 04:55:07,872 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/.tmp/cf1/ba881bf138a5441cb5b93140d550ada5 as hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1/ba881bf138a5441cb5b93140d550ada5 2023-07-18 04:55:07,877 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/info/ef71435e7c674e028950867c5770cbd2, entries=24, sequenceid=43, filesize=7.4 K 2023-07-18 04:55:07,878 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 48ms, sequenceid=43, compaction requested=true 2023-07-18 04:55:07,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/cf1/ba881bf138a5441cb5b93140d550ada5, entries=100, sequenceid=21, filesize=8.3 K 2023-07-18 04:55:07,891 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for a2e71ceab1ce1a583f43e863ff7cd0a0 in 76ms, sequenceid=21, compaction requested=true 2023-07-18 04:55:07,901 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/meta/1588230740/recovered.edits/46.seqid, newMaxSeqId=46, maxSeqId=34 2023-07-18 04:55:07,902 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-18 04:55:07,902 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-18 04:55:07,902 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-18 04:55:07,902 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-18 04:55:07,903 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t1_syncup/a2e71ceab1ce1a583f43e863ff7cd0a0/recovered.edits/24.seqid, newMaxSeqId=24, maxSeqId=17 2023-07-18 04:55:07,904 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:55:07,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a2e71ceab1ce1a583f43e863ff7cd0a0: 2023-07-18 04:55:07,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689656060740.a2e71ceab1ce1a583f43e863ff7cd0a0. 2023-07-18 04:55:07,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1b9326fa16ae8497bb1df12a88dfd577, disabling compactions & flushes 2023-07-18 04:55:07,905 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:55:07,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:55:07,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. after waiting 0 ms 2023-07-18 04:55:07,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:55:07,905 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1b9326fa16ae8497bb1df12a88dfd577 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-18 04:55:07,917 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/.tmp/cf1/cff83f519cb94376a02f1cd63c865642 2023-07-18 04:55:07,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/.tmp/cf1/cff83f519cb94376a02f1cd63c865642 as hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1/cff83f519cb94376a02f1cd63c865642 2023-07-18 04:55:07,928 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/cf1/cff83f519cb94376a02f1cd63c865642, entries=200, sequenceid=22, filesize=12.2 K 2023-07-18 04:55:07,929 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for 1b9326fa16ae8497bb1df12a88dfd577 in 24ms, sequenceid=22, compaction requested=true 2023-07-18 04:55:07,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/default/t2_syncup/1b9326fa16ae8497bb1df12a88dfd577/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=18 2023-07-18 04:55:07,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:55:07,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1b9326fa16ae8497bb1df12a88dfd577: 2023-07-18 04:55:07,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689656061358.1b9326fa16ae8497bb1df12a88dfd577. 2023-07-18 04:55:07,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 383bb30520997d1bb825dc5e5f31999e, disabling compactions & flushes 2023-07-18 04:55:07,942 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:55:07,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:55:07,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. after waiting 0 ms 2023-07-18 04:55:07,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:55:07,951 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/data/hbase/namespace/383bb30520997d1bb825dc5e5f31999e/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=12 2023-07-18 04:55:07,952 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:55:07,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 383bb30520997d1bb825dc5e5f31999e: 2023-07-18 04:55:07,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689656058840.383bb30520997d1bb825dc5e5f31999e. 2023-07-18 04:55:08,030 INFO [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34513,1689656091182; all regions closed. 2023-07-18 04:55:08,030 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:55:08,039 DEBUG [RS:0;jenkins-hbase4:34513] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs 2023-07-18 04:55:08,039 INFO [RS:0;jenkins-hbase4:34513] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C34513%2C1689656091182.meta:.meta(num 1689656091686) 2023-07-18 04:55:08,039 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:55:08,046 DEBUG [RS:0;jenkins-hbase4:34513] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs 2023-07-18 04:55:08,046 INFO [RS:0;jenkins-hbase4:34513] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C34513%2C1689656091182.jenkins-hbase4.apache.org%2C34513%2C1689656091182.regiongroup-1:(num 1689656096075) 2023-07-18 04:55:08,046 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/WALs/jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:55:08,053 DEBUG [RS:0;jenkins-hbase4:34513] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/oldWALs 2023-07-18 04:55:08,053 INFO [RS:0;jenkins-hbase4:34513] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C34513%2C1689656091182.jenkins-hbase4.apache.org%2C34513%2C1689656091182.regiongroup-0:(num 1689656091594) 2023-07-18 04:55:08,053 DEBUG [RS:0;jenkins-hbase4:34513] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:55:08,053 INFO [RS:0;jenkins-hbase4:34513] regionserver.LeaseManager(133): Closed leases 2023-07-18 04:55:08,053 INFO [RS:0;jenkins-hbase4:34513] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-18 04:55:08,053 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 04:55:08,053 DEBUG [RS:0;jenkins-hbase4:34513] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x447a1cd0 to 127.0.0.1:63236 2023-07-18 04:55:08,053 DEBUG [RS:0;jenkins-hbase4:34513] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:55:08,054 INFO [RS:0;jenkins-hbase4:34513] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34513 2023-07-18 04:55:08,059 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:34513-0x10176c7bf3c0031, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,34513,1689656091182 2023-07-18 04:55:08,059 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10176c7bf3c0026, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:55:08,060 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:34513-0x10176c7bf3c0031, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:55:08,060 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 04:55:08,060 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10176c7bf3c003a, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:55:08,059 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-18 04:55:08,060 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-18 04:55:08,061 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34513,1689656091182] 2023-07-18 04:55:08,061 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34513,1689656091182; numProcessing=1 2023-07-18 04:55:08,062 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,34513,1689656091182 already deleted, retry=false 2023-07-18 04:55:08,063 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34513,1689656091182 expired; onlineServers=0 2023-07-18 04:55:08,063 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41819,1689656091150' ***** 2023-07-18 04:55:08,063 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-18 04:55:08,063 DEBUG [M:0;jenkins-hbase4:41819] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4d962aa2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-18 04:55:08,063 INFO [M:0;jenkins-hbase4:41819] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41819,1689656091150 2023-07-18 04:55:08,063 INFO [M:0;jenkins-hbase4:41819] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41819,1689656091150; all regions closed. 2023-07-18 04:55:08,063 DEBUG [M:0;jenkins-hbase4:41819] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-18 04:55:08,063 DEBUG [M:0;jenkins-hbase4:41819] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-18 04:55:08,064 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-18 04:55:08,064 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656091356] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689656091356,5,FailOnTimeoutGroup] 2023-07-18 04:55:08,064 DEBUG [M:0;jenkins-hbase4:41819] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-18 04:55:08,064 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656091355] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689656091355,5,FailOnTimeoutGroup] 2023-07-18 04:55:08,064 INFO [M:0;jenkins-hbase4:41819] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-18 04:55:08,065 INFO [M:0;jenkins-hbase4:41819] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-18 04:55:08,065 INFO [M:0;jenkins-hbase4:41819] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-18 04:55:08,065 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-18 04:55:08,065 DEBUG [M:0;jenkins-hbase4:41819] master.HMaster(1512): Stopping service threads 2023-07-18 04:55:08,065 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-18 04:55:08,065 INFO [M:0;jenkins-hbase4:41819] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-18 04:55:08,066 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-18 04:55:08,066 ERROR [M:0;jenkins-hbase4:41819] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-18 04:55:08,066 INFO [M:0;jenkins-hbase4:41819] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-18 04:55:08,066 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-18 04:55:08,067 DEBUG [M:0;jenkins-hbase4:41819] zookeeper.ZKUtil(398): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-18 04:55:08,067 WARN [M:0;jenkins-hbase4:41819] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-18 04:55:08,067 INFO [M:0;jenkins-hbase4:41819] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-18 04:55:08,067 INFO [M:0;jenkins-hbase4:41819] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-18 04:55:08,068 DEBUG [M:0;jenkins-hbase4:41819] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-18 04:55:08,068 INFO [M:0;jenkins-hbase4:41819] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:55:08,068 DEBUG [M:0;jenkins-hbase4:41819] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:55:08,068 DEBUG [M:0;jenkins-hbase4:41819] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-18 04:55:08,068 DEBUG [M:0;jenkins-hbase4:41819] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:55:08,068 INFO [M:0;jenkins-hbase4:41819] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=26.42 KB heapSize=32.85 KB 2023-07-18 04:55:08,085 INFO [M:0;jenkins-hbase4:41819] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=26.42 KB at sequenceid=288 (bloomFilter=true), to=hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a8fb22bb1f2b4ff8893dcb6f8f83a51f 2023-07-18 04:55:08,091 DEBUG [M:0;jenkins-hbase4:41819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a8fb22bb1f2b4ff8893dcb6f8f83a51f as hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a8fb22bb1f2b4ff8893dcb6f8f83a51f 2023-07-18 04:55:08,097 INFO [M:0;jenkins-hbase4:41819] regionserver.HStore(1080): Added hdfs://localhost:37939/user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a8fb22bb1f2b4ff8893dcb6f8f83a51f, entries=9, sequenceid=288, filesize=5.5 K 2023-07-18 04:55:08,098 INFO [M:0;jenkins-hbase4:41819] regionserver.HRegion(2948): Finished flush of dataSize ~26.42 KB/27051, heapSize ~32.84 KB/33624, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 30ms, sequenceid=288, compaction requested=true 2023-07-18 04:55:08,100 INFO [M:0;jenkins-hbase4:41819] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-18 04:55:08,100 DEBUG [M:0;jenkins-hbase4:41819] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-18 04:55:08,100 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/0f84940b-a1cb-ee29-32f9-af3f7c64d3a0/MasterData/WALs/jenkins-hbase4.apache.org,41819,1689656091150 2023-07-18 04:55:08,108 INFO [M:0;jenkins-hbase4:41819] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-18 04:55:08,108 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-18 04:55:08,108 INFO [M:0;jenkins-hbase4:41819] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41819 2023-07-18 04:55:08,110 DEBUG [M:0;jenkins-hbase4:41819] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,41819,1689656091150 already deleted, retry=false 2023-07-18 04:55:08,195 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-18 04:55:08,300 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:55:08,300 INFO [M:0;jenkins-hbase4:41819] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41819,1689656091150; zookeeper connection closed. 2023-07-18 04:55:08,301 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): master:41819-0x10176c7bf3c0030, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:55:08,401 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:34513-0x10176c7bf3c0031, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:55:08,401 INFO [RS:0;jenkins-hbase4:34513] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34513,1689656091182; zookeeper connection closed. 2023-07-18 04:55:08,401 DEBUG [Listener at localhost/37011-EventThread] zookeeper.ZKWatcher(600): regionserver:34513-0x10176c7bf3c0031, quorum=127.0.0.1:63236, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-18 04:55:08,402 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@54c7c208] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@54c7c208 2023-07-18 04:55:08,402 INFO [Listener at localhost/37011] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-18 04:55:08,403 WARN [Listener at localhost/37011] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-18 04:55:08,408 INFO [Listener at localhost/37011] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 04:55:08,515 WARN [BP-433638424-172.31.14.131-1689656056846 heartbeating to localhost/127.0.0.1:37939] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-18 04:55:08,515 WARN [BP-433638424-172.31.14.131-1689656056846 heartbeating to localhost/127.0.0.1:37939] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-433638424-172.31.14.131-1689656056846 (Datanode Uuid a496e3f3-5645-4243-8cf8-1af079891093) service to localhost/127.0.0.1:37939 2023-07-18 04:55:08,518 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/cluster_328436eb-65ac-c7b0-f033-ec4bcc6042f1/dfs/data/data7/current/BP-433638424-172.31.14.131-1689656056846] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 04:55:08,518 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/cluster_328436eb-65ac-c7b0-f033-ec4bcc6042f1/dfs/data/data8/current/BP-433638424-172.31.14.131-1689656056846] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 04:55:08,521 WARN [Listener at localhost/37011] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-18 04:55:08,525 INFO [Listener at localhost/37011] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 04:55:08,632 WARN [BP-433638424-172.31.14.131-1689656056846 heartbeating to localhost/127.0.0.1:37939] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-18 04:55:08,632 WARN [BP-433638424-172.31.14.131-1689656056846 heartbeating to localhost/127.0.0.1:37939] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-433638424-172.31.14.131-1689656056846 (Datanode Uuid 17506207-b0e0-4651-9028-781a6610874c) service to localhost/127.0.0.1:37939 2023-07-18 04:55:08,633 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/cluster_328436eb-65ac-c7b0-f033-ec4bcc6042f1/dfs/data/data5/current/BP-433638424-172.31.14.131-1689656056846] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 04:55:08,633 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/cluster_328436eb-65ac-c7b0-f033-ec4bcc6042f1/dfs/data/data6/current/BP-433638424-172.31.14.131-1689656056846] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 04:55:08,635 WARN [Listener at localhost/37011] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-18 04:55:08,638 INFO [Listener at localhost/37011] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 04:55:08,743 WARN [BP-433638424-172.31.14.131-1689656056846 heartbeating to localhost/127.0.0.1:37939] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-18 04:55:08,743 WARN [BP-433638424-172.31.14.131-1689656056846 heartbeating to localhost/127.0.0.1:37939] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-433638424-172.31.14.131-1689656056846 (Datanode Uuid 7e19e0ad-89aa-4293-a01b-8c1ee631c073) service to localhost/127.0.0.1:37939 2023-07-18 04:55:08,744 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/cluster_328436eb-65ac-c7b0-f033-ec4bcc6042f1/dfs/data/data3/current/BP-433638424-172.31.14.131-1689656056846] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 04:55:08,744 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/cluster_328436eb-65ac-c7b0-f033-ec4bcc6042f1/dfs/data/data4/current/BP-433638424-172.31.14.131-1689656056846] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 04:55:08,746 WARN [Listener at localhost/37011] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-18 04:55:08,750 INFO [Listener at localhost/37011] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 04:55:08,856 WARN [BP-433638424-172.31.14.131-1689656056846 heartbeating to localhost/127.0.0.1:37939] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-18 04:55:08,857 WARN [BP-433638424-172.31.14.131-1689656056846 heartbeating to localhost/127.0.0.1:37939] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-433638424-172.31.14.131-1689656056846 (Datanode Uuid e06ca105-963d-4852-b6fe-f6a9b0a5e830) service to localhost/127.0.0.1:37939 2023-07-18 04:55:08,857 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/cluster_328436eb-65ac-c7b0-f033-ec4bcc6042f1/dfs/data/data1/current/BP-433638424-172.31.14.131-1689656056846] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 04:55:08,858 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/04b7b805-1ebe-8471-3edf-42e38b653219/cluster_328436eb-65ac-c7b0-f033-ec4bcc6042f1/dfs/data/data2/current/BP-433638424-172.31.14.131-1689656056846] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 04:55:08,891 INFO [Listener at localhost/37011] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 04:55:09,066 INFO [Listener at localhost/37011] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-18 04:55:09,066 INFO [Listener at localhost/37011] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-18 04:55:09,066 WARN [Listener at localhost/37011] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-18 04:55:09,068 INFO [Listener at localhost/37011] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 04:55:09,169 WARN [BP-527541699-172.31.14.131-1689656049856 heartbeating to localhost/127.0.0.1:32937] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-18 04:55:09,169 WARN [BP-527541699-172.31.14.131-1689656049856 heartbeating to localhost/127.0.0.1:32937] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-527541699-172.31.14.131-1689656049856 (Datanode Uuid 1403a88c-ea03-40ca-b217-fac66b980697) service to localhost/127.0.0.1:32937 2023-07-18 04:55:09,170 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/cluster_33cb87ab-e4a2-016e-8b73-ae4d9306366b/dfs/data/data3/current/BP-527541699-172.31.14.131-1689656049856] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 04:55:09,171 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/cluster_33cb87ab-e4a2-016e-8b73-ae4d9306366b/dfs/data/data4/current/BP-527541699-172.31.14.131-1689656049856] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 04:55:09,171 WARN [Listener at localhost/37011] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-18 04:55:09,173 INFO [Listener at localhost/37011] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 04:55:09,274 WARN [BP-527541699-172.31.14.131-1689656049856 heartbeating to localhost/127.0.0.1:32937] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-18 04:55:09,275 WARN [BP-527541699-172.31.14.131-1689656049856 heartbeating to localhost/127.0.0.1:32937] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-527541699-172.31.14.131-1689656049856 (Datanode Uuid d7b9589a-9b20-4c9b-be4d-dff0a025932b) service to localhost/127.0.0.1:32937 2023-07-18 04:55:09,275 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/cluster_33cb87ab-e4a2-016e-8b73-ae4d9306366b/dfs/data/data1/current/BP-527541699-172.31.14.131-1689656049856] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 04:55:09,276 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f8fbeb9-feb6-17af-519e-21e9af70c3f0/cluster_33cb87ab-e4a2-016e-8b73-ae4d9306366b/dfs/data/data2/current/BP-527541699-172.31.14.131-1689656049856] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-18 04:55:09,284 INFO [Listener at localhost/37011] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-18 04:55:09,399 INFO [Listener at localhost/37011] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-18 04:55:09,425 INFO [Listener at localhost/37011] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-18 04:55:09,447 INFO [Listener at localhost/37011] hbase.ResourceChecker(175): after: replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL#testSyncUpTool Thread=182 (was 10) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:32937 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:37939 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-15 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63236@0x3c29d558 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1497514064.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63236@0x272235ca-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-9 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144-SendThread(127.0.0.1:63236) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-11-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:37939 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63236@0x1142dfc9-SendThread(127.0.0.1:63236) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: IPC Client (786611744) connection to localhost/127.0.0.1:37939 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost:37939 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost:37939 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost:32937 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: RS-EventLoopGroup-14-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63236@0x272235ca sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1497514064.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/37011 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63236@0x65cff136-SendThread(127.0.0.1:63236) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (786611744) connection to localhost/127.0.0.1:32937 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-13-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-13 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:37939 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:32937 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190-SendThread(127.0.0.1:63236) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45857,1689656054144-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:37939 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63236@0x65cff136 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1497514064.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:37939 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (786611744) connection to localhost/127.0.0.1:37939 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63236@0x272235ca-SendThread(127.0.0.1:63236) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63236@0x1142dfc9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1497514064.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63236@0x3c29d558-SendThread(127.0.0.1:63236) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:228) org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1338) org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1276) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1254) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-14 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-11 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63236@0x1142dfc9-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-10 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,45615,1689656083190-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: LeaseRenewer:jenkins@localhost:32937 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (786611744) connection to localhost/127.0.0.1:37939 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63236@0x65cff136-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63236@0x3c29d558-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@76f22418 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (786611744) connection to localhost/127.0.0.1:32937 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-12 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=692 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=157 (was 229), ProcessCount=170 (was 173), AvailableMemoryMB=5936 (was 7044)