2023-07-20 16:54:15,714 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8 2023-07-20 16:54:15,722 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/hadoop-log-dir so I do NOT create it in target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503 2023-07-20 16:54:15,723 WARN [main] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/hadoop-log-dir Erasing configuration value by system value. 2023-07-20 16:54:15,723 INFO [main] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/hadoop-tmp-dir so I do NOT create it in target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503 2023-07-20 16:54:15,723 WARN [main] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-20 16:54:15,723 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503 2023-07-20 16:54:15,736 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool timeout: 13 mins 2023-07-20 16:54:15,737 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL timeout: 13 mins 2023-07-20 16:54:15,767 INFO [Time-limited test] hbase.ResourceChecker(147): before: replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL#testSyncUpTool Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=268, ProcessCount=173, AvailableMemoryMB=6717 2023-07-20 16:54:15,778 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/hadoop-log-dir so I do NOT create it in target/test-data/dce6438b-9665-7334-c7b2-71e73d958089 2023-07-20 16:54:15,778 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/hadoop-log-dir Erasing configuration value by system value. 2023-07-20 16:54:15,778 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/hadoop-tmp-dir so I do NOT create it in target/test-data/dce6438b-9665-7334-c7b2-71e73d958089 2023-07-20 16:54:15,778 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-20 16:54:15,779 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dce6438b-9665-7334-c7b2-71e73d958089 2023-07-20 16:54:15,788 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/hadoop-log-dir so I do NOT create it in target/test-data/1500d5bc-b330-9285-1c15-3d1ed4bcdeed 2023-07-20 16:54:15,789 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/hadoop-log-dir Erasing configuration value by system value. 2023-07-20 16:54:15,789 INFO [Time-limited test] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/hadoop-tmp-dir so I do NOT create it in target/test-data/1500d5bc-b330-9285-1c15-3d1ed4bcdeed 2023-07-20 16:54:15,789 WARN [Time-limited test] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/hadoop-tmp-dir Erasing configuration value by system value. 2023-07-20 16:54:15,790 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1500d5bc-b330-9285-1c15-3d1ed4bcdeed 2023-07-20 16:54:15,793 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/cluster_1bbf2ea4-f49d-7f4b-90b2-cfdea3662c1b, deleteOnExit=true 2023-07-20 16:54:15,910 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/cluster_1bbf2ea4-f49d-7f4b-90b2-cfdea3662c1b/zookeeper_0, clientPort=56032, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/cluster_1bbf2ea4-f49d-7f4b-90b2-cfdea3662c1b/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/cluster_1bbf2ea4-f49d-7f4b-90b2-cfdea3662c1b/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-20 16:54:15,925 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=56032 2023-07-20 16:54:15,928 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=2, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-20 16:54:15,929 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-20 16:54:15,929 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/test.cache.data in system properties and HBase conf 2023-07-20 16:54:15,930 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/hadoop.tmp.dir in system properties and HBase conf 2023-07-20 16:54:15,930 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/hadoop.log.dir in system properties and HBase conf 2023-07-20 16:54:15,931 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-20 16:54:15,931 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-20 16:54:15,932 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-20 16:54:16,034 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-20 16:54:16,430 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-20 16:54:16,434 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-20 16:54:16,435 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-20 16:54:16,435 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-20 16:54:16,435 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-20 16:54:16,435 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-20 16:54:16,436 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-20 16:54:16,436 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-20 16:54:16,436 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-20 16:54:16,436 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-20 16:54:16,437 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/nfs.dump.dir in system properties and HBase conf 2023-07-20 16:54:16,437 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/java.io.tmpdir in system properties and HBase conf 2023-07-20 16:54:16,437 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-20 16:54:16,437 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-20 16:54:16,438 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-20 16:54:16,936 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-20 16:54:16,940 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-20 16:54:17,236 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-20 16:54:17,373 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-20 16:54:17,386 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 16:54:17,423 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 16:54:17,456 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/java.io.tmpdir/Jetty_localhost_37509_hdfs____1vvh6s/webapp 2023-07-20 16:54:17,589 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37509 2023-07-20 16:54:17,610 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-20 16:54:17,611 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-20 16:54:18,144 WARN [Listener at localhost/38055] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 16:54:18,211 WARN [Listener at localhost/38055] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-20 16:54:18,231 WARN [Listener at localhost/38055] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 16:54:18,238 INFO [Listener at localhost/38055] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 16:54:18,242 INFO [Listener at localhost/38055] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/java.io.tmpdir/Jetty_localhost_35435_datanode____.12och1/webapp 2023-07-20 16:54:18,345 INFO [Listener at localhost/38055] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35435 2023-07-20 16:54:18,622 WARN [Listener at localhost/40869] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 16:54:18,635 WARN [Listener at localhost/40869] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-20 16:54:18,639 WARN [Listener at localhost/40869] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 16:54:18,641 INFO [Listener at localhost/40869] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 16:54:18,649 INFO [Listener at localhost/40869] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/java.io.tmpdir/Jetty_localhost_38703_datanode____c5d5r0/webapp 2023-07-20 16:54:18,751 INFO [Listener at localhost/40869] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38703 2023-07-20 16:54:18,762 WARN [Listener at localhost/36209] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 16:54:19,084 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x65b14f0daac7cfeb: Processing first storage report for DS-87ab81e2-6016-492e-a423-b308cc471afd from datanode b50e5c74-ff31-44ab-9bcf-8c87552d91c8 2023-07-20 16:54:19,086 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x65b14f0daac7cfeb: from storage DS-87ab81e2-6016-492e-a423-b308cc471afd node DatanodeRegistration(127.0.0.1:45027, datanodeUuid=b50e5c74-ff31-44ab-9bcf-8c87552d91c8, infoPort=34953, infoSecurePort=0, ipcPort=40869, storageInfo=lv=-57;cid=testClusterID;nsid=1222323026;c=1689872057011), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-07-20 16:54:19,086 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x51fca96e4326da39: Processing first storage report for DS-82a10023-4ecc-44e1-b86d-cd4749f69898 from datanode 616ed7c7-88a2-4fe7-8fad-43f3c94c5211 2023-07-20 16:54:19,086 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x51fca96e4326da39: from storage DS-82a10023-4ecc-44e1-b86d-cd4749f69898 node DatanodeRegistration(127.0.0.1:40699, datanodeUuid=616ed7c7-88a2-4fe7-8fad-43f3c94c5211, infoPort=35555, infoSecurePort=0, ipcPort=36209, storageInfo=lv=-57;cid=testClusterID;nsid=1222323026;c=1689872057011), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 16:54:19,086 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x65b14f0daac7cfeb: Processing first storage report for DS-dcd1be4f-dd34-48bd-b93f-513591e43ad0 from datanode b50e5c74-ff31-44ab-9bcf-8c87552d91c8 2023-07-20 16:54:19,087 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x65b14f0daac7cfeb: from storage DS-dcd1be4f-dd34-48bd-b93f-513591e43ad0 node DatanodeRegistration(127.0.0.1:45027, datanodeUuid=b50e5c74-ff31-44ab-9bcf-8c87552d91c8, infoPort=34953, infoSecurePort=0, ipcPort=40869, storageInfo=lv=-57;cid=testClusterID;nsid=1222323026;c=1689872057011), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 16:54:19,087 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x51fca96e4326da39: Processing first storage report for DS-48d7f2ac-59b4-45c1-adfd-212306694d7a from datanode 616ed7c7-88a2-4fe7-8fad-43f3c94c5211 2023-07-20 16:54:19,087 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x51fca96e4326da39: from storage DS-48d7f2ac-59b4-45c1-adfd-212306694d7a node DatanodeRegistration(127.0.0.1:40699, datanodeUuid=616ed7c7-88a2-4fe7-8fad-43f3c94c5211, infoPort=35555, infoSecurePort=0, ipcPort=36209, storageInfo=lv=-57;cid=testClusterID;nsid=1222323026;c=1689872057011), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 16:54:19,199 DEBUG [Listener at localhost/36209] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8 2023-07-20 16:54:19,208 INFO [Listener at localhost/36209] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:19,210 INFO [Listener at localhost/36209] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:19,899 INFO [Listener at localhost/36209] util.FSUtils(471): Created version file at hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738 with version=8 2023-07-20 16:54:19,900 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/hbase-staging 2023-07-20 16:54:20,233 INFO [Listener at localhost/36209] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-20 16:54:20,734 INFO [Listener at localhost/36209] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-20 16:54:20,766 INFO [Listener at localhost/36209] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:20,766 INFO [Listener at localhost/36209] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:20,766 INFO [Listener at localhost/36209] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 16:54:20,767 INFO [Listener at localhost/36209] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:20,767 INFO [Listener at localhost/36209] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 16:54:20,922 INFO [Listener at localhost/36209] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 16:54:20,998 DEBUG [Listener at localhost/36209] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-20 16:54:21,097 INFO [Listener at localhost/36209] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39639 2023-07-20 16:54:21,108 INFO [Listener at localhost/36209] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:21,111 INFO [Listener at localhost/36209] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:21,133 INFO [Listener at localhost/36209] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39639 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:21,179 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:396390x0, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:21,181 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:39639-0x10183a7c1310000 connected 2023-07-20 16:54:21,213 DEBUG [Listener at localhost/36209] zookeeper.ZKUtil(164): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-20 16:54:21,214 DEBUG [Listener at localhost/36209] zookeeper.ZKUtil(164): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 16:54:21,218 DEBUG [Listener at localhost/36209] zookeeper.ZKUtil(164): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-20 16:54:21,228 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39639 2023-07-20 16:54:21,229 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39639 2023-07-20 16:54:21,229 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39639 2023-07-20 16:54:21,230 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39639 2023-07-20 16:54:21,230 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39639 2023-07-20 16:54:21,235 INFO [Listener at localhost/36209] master.HMaster(444): hbase.rootdir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738, hbase.cluster.distributed=false 2023-07-20 16:54:21,306 INFO [Listener at localhost/36209] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-20 16:54:21,306 INFO [Listener at localhost/36209] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:21,306 INFO [Listener at localhost/36209] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:21,306 INFO [Listener at localhost/36209] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 16:54:21,306 INFO [Listener at localhost/36209] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:21,307 INFO [Listener at localhost/36209] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 16:54:21,312 INFO [Listener at localhost/36209] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 16:54:21,314 INFO [Listener at localhost/36209] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33103 2023-07-20 16:54:21,317 INFO [Listener at localhost/36209] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 16:54:21,323 DEBUG [Listener at localhost/36209] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 16:54:21,324 INFO [Listener at localhost/36209] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:21,327 INFO [Listener at localhost/36209] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:21,329 INFO [Listener at localhost/36209] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33103 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:21,332 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:331030x0, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:21,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:33103-0x10183a7c1310001 connected 2023-07-20 16:54:21,333 DEBUG [Listener at localhost/36209] zookeeper.ZKUtil(164): regionserver:33103-0x10183a7c1310001, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-20 16:54:21,334 DEBUG [Listener at localhost/36209] zookeeper.ZKUtil(164): regionserver:33103-0x10183a7c1310001, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 16:54:21,335 DEBUG [Listener at localhost/36209] zookeeper.ZKUtil(164): regionserver:33103-0x10183a7c1310001, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-20 16:54:21,342 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33103 2023-07-20 16:54:21,342 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33103 2023-07-20 16:54:21,342 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33103 2023-07-20 16:54:21,343 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33103 2023-07-20 16:54:21,343 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33103 2023-07-20 16:54:21,358 INFO [Listener at localhost/36209] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-20 16:54:21,358 INFO [Listener at localhost/36209] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:21,359 INFO [Listener at localhost/36209] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:21,359 INFO [Listener at localhost/36209] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 16:54:21,360 INFO [Listener at localhost/36209] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:21,360 INFO [Listener at localhost/36209] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 16:54:21,360 INFO [Listener at localhost/36209] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 16:54:21,362 INFO [Listener at localhost/36209] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34183 2023-07-20 16:54:21,362 INFO [Listener at localhost/36209] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 16:54:21,364 DEBUG [Listener at localhost/36209] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 16:54:21,365 INFO [Listener at localhost/36209] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:21,367 INFO [Listener at localhost/36209] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:21,368 INFO [Listener at localhost/36209] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34183 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:21,375 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:341830x0, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:21,376 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34183-0x10183a7c1310002 connected 2023-07-20 16:54:21,377 DEBUG [Listener at localhost/36209] zookeeper.ZKUtil(164): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-20 16:54:21,377 DEBUG [Listener at localhost/36209] zookeeper.ZKUtil(164): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 16:54:21,378 DEBUG [Listener at localhost/36209] zookeeper.ZKUtil(164): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-20 16:54:21,382 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34183 2023-07-20 16:54:21,386 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34183 2023-07-20 16:54:21,387 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34183 2023-07-20 16:54:21,387 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34183 2023-07-20 16:54:21,388 DEBUG [Listener at localhost/36209] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34183 2023-07-20 16:54:21,389 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,39639,1689872060057 2023-07-20 16:54:21,398 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-20 16:54:21,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,39639,1689872060057 2023-07-20 16:54:21,423 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-20 16:54:21,423 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-20 16:54:21,423 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:33103-0x10183a7c1310001, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-20 16:54:21,423 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:54:21,425 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-20 16:54:21,426 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,39639,1689872060057 from backup master directory 2023-07-20 16:54:21,426 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-20 16:54:21,429 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,39639,1689872060057 2023-07-20 16:54:21,429 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-20 16:54:21,430 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 16:54:21,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,39639,1689872060057 2023-07-20 16:54:21,433 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-20 16:54:21,434 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-20 16:54:21,538 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/hbase.id with ID: 258804e0-584b-4fe6-b46a-945d0467644e 2023-07-20 16:54:21,590 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:21,611 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:54:21,656 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2531dee3 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:21,690 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1cfcbc51, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:21,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 16:54:21,716 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-20 16:54:21,725 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:54:21,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:54:21,762 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store-tmp 2023-07-20 16:54:21,800 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:21,800 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-20 16:54:21,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:21,801 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:21,801 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-20 16:54:21,801 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:21,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:21,801 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 16:54:21,802 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/WALs/jenkins-hbase4.apache.org,39639,1689872060057 2023-07-20 16:54:21,805 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:21,826 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39639%2C1689872060057.jenkins-hbase4.apache.org%2C39639%2C1689872060057.regiongroup-0, suffix=, logDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/WALs/jenkins-hbase4.apache.org,39639,1689872060057, archiveDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/oldWALs, maxLogs=10 2023-07-20 16:54:21,846 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.CommonFSUtils$DfsBuilderUtility(753): Could not find replicate method on builder; will not set replicate when creating output stream java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DistributedFileSystem$HdfsDataOutputStreamBuilder.replicate() at java.lang.Class.getMethod(Class.java:1786) at org.apache.hadoop.hbase.util.CommonFSUtils$DfsBuilderUtility.(CommonFSUtils.java:750) at org.apache.hadoop.hbase.util.CommonFSUtils.createForWal(CommonFSUtils.java:802) at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.initOutput(ProtobufLogWriter.java:102) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.FSHLogProvider.createWriter(FSHLogProvider.java:78) at org.apache.hadoop.hbase.regionserver.wal.FSHLog.createWriterInstance(FSHLog.java:307) at org.apache.hadoop.hbase.regionserver.wal.FSHLog.createWriterInstance(FSHLog.java:70) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:195) at org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:215) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-20 16:54:21,872 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/WALs/jenkins-hbase4.apache.org,39639,1689872060057/jenkins-hbase4.apache.org%2C39639%2C1689872060057.jenkins-hbase4.apache.org%2C39639%2C1689872060057.regiongroup-0.1689872061844 2023-07-20 16:54:21,872 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45027,DS-87ab81e2-6016-492e-a423-b308cc471afd,DISK], DatanodeInfoWithStorage[127.0.0.1:40699,DS-82a10023-4ecc-44e1-b86d-cd4749f69898,DISK]] 2023-07-20 16:54:21,873 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:21,873 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:21,877 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:21,878 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:21,934 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:21,940 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-20 16:54:21,964 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-20 16:54:21,977 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:21,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:21,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:22,000 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:22,005 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 16:54:22,006 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67855962, jitterRate=0.011132627725601196}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 16:54:22,006 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 16:54:22,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-20 16:54:22,028 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-20 16:54:22,028 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-20 16:54:22,031 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-20 16:54:22,035 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 3 msec 2023-07-20 16:54:22,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 33 msec 2023-07-20 16:54:22,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-20 16:54:22,094 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-20 16:54:22,101 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-20 16:54:22,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-20 16:54:22,134 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-20 16:54:22,136 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-20 16:54:22,141 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-20 16:54:22,145 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-20 16:54:22,148 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:54:22,150 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-20 16:54:22,150 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-20 16:54:22,163 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-20 16:54:22,168 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-20 16:54:22,168 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-20 16:54:22,168 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:33103-0x10183a7c1310001, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-20 16:54:22,168 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:54:22,169 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,39639,1689872060057, sessionid=0x10183a7c1310000, setting cluster-up flag (Was=false) 2023-07-20 16:54:22,186 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:54:22,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-20 16:54:22,195 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39639,1689872060057 2023-07-20 16:54:22,200 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:54:22,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-20 16:54:22,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39639,1689872060057 2023-07-20 16:54:22,210 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.hbase-snapshot/.tmp 2023-07-20 16:54:22,291 INFO [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(951): ClusterId : 258804e0-584b-4fe6-b46a-945d0467644e 2023-07-20 16:54:22,291 INFO [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(951): ClusterId : 258804e0-584b-4fe6-b46a-945d0467644e 2023-07-20 16:54:22,296 DEBUG [RS:0;jenkins-hbase4:33103] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 16:54:22,296 DEBUG [RS:1;jenkins-hbase4:34183] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 16:54:22,303 DEBUG [RS:1;jenkins-hbase4:34183] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 16:54:22,303 DEBUG [RS:0;jenkins-hbase4:33103] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 16:54:22,304 DEBUG [RS:1;jenkins-hbase4:34183] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 16:54:22,304 DEBUG [RS:0;jenkins-hbase4:33103] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 16:54:22,310 DEBUG [RS:1;jenkins-hbase4:34183] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 16:54:22,310 DEBUG [RS:0;jenkins-hbase4:33103] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 16:54:22,312 DEBUG [RS:1;jenkins-hbase4:34183] zookeeper.ReadOnlyZKClient(139): Connect 0x03c82af4 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:22,312 DEBUG [RS:0;jenkins-hbase4:33103] zookeeper.ReadOnlyZKClient(139): Connect 0x1290d046 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:22,319 DEBUG [RS:0;jenkins-hbase4:33103] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c572c64, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:22,320 DEBUG [RS:0;jenkins-hbase4:33103] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1868bb36, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 16:54:22,320 DEBUG [RS:1;jenkins-hbase4:34183] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@d0fc4fd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:22,320 DEBUG [RS:1;jenkins-hbase4:34183] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@9304184, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 16:54:22,326 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-20 16:54:22,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:54:22,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:54:22,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:54:22,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:54:22,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-20 16:54:22,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 16:54:22,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,341 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689872092341 2023-07-20 16:54:22,345 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-20 16:54:22,351 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-20 16:54:22,352 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-20 16:54:22,357 DEBUG [RS:0;jenkins-hbase4:33103] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:33103 2023-07-20 16:54:22,357 DEBUG [RS:1;jenkins-hbase4:34183] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:34183 2023-07-20 16:54:22,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-20 16:54:22,359 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-20 16:54:22,362 INFO [RS:0;jenkins-hbase4:33103] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 16:54:22,362 INFO [RS:1;jenkins-hbase4:34183] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 16:54:22,363 INFO [RS:0;jenkins-hbase4:33103] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 16:54:22,364 INFO [RS:1;jenkins-hbase4:34183] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 16:54:22,364 DEBUG [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 16:54:22,364 DEBUG [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 16:54:22,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-20 16:54:22,368 INFO [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39639,1689872060057 with isa=jenkins-hbase4.apache.org/172.31.14.131:34183, startcode=1689872061357 2023-07-20 16:54:22,368 INFO [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39639,1689872060057 with isa=jenkins-hbase4.apache.org/172.31.14.131:33103, startcode=1689872061305 2023-07-20 16:54:22,368 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-20 16:54:22,368 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-20 16:54:22,368 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-20 16:54:22,369 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,371 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-20 16:54:22,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-20 16:54:22,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-20 16:54:22,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-20 16:54:22,378 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-20 16:54:22,381 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872062380,5,FailOnTimeoutGroup] 2023-07-20 16:54:22,382 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872062382,5,FailOnTimeoutGroup] 2023-07-20 16:54:22,382 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,382 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-20 16:54:22,384 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,384 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,394 DEBUG [RS:0;jenkins-hbase4:33103] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 16:54:22,394 DEBUG [RS:1;jenkins-hbase4:34183] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 16:54:22,410 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-20 16:54:22,412 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-20 16:54:22,412 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738 2023-07-20 16:54:22,444 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:22,447 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-20 16:54:22,450 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/info 2023-07-20 16:54:22,451 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-20 16:54:22,453 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:22,453 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-20 16:54:22,456 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/rep_barrier 2023-07-20 16:54:22,456 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-20 16:54:22,457 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:22,457 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-20 16:54:22,463 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/table 2023-07-20 16:54:22,463 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-20 16:54:22,464 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:22,466 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740 2023-07-20 16:54:22,467 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740 2023-07-20 16:54:22,472 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-20 16:54:22,474 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-20 16:54:22,483 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 16:54:22,485 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60443931, jitterRate=-0.09931524097919464}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-20 16:54:22,485 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-20 16:54:22,485 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-20 16:54:22,485 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-20 16:54:22,485 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-20 16:54:22,486 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-20 16:54:22,486 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-20 16:54:22,487 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-20 16:54:22,488 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-20 16:54:22,495 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-20 16:54:22,495 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-20 16:54:22,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-20 16:54:22,527 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-20 16:54:22,530 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-20 16:54:22,541 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44887, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 16:54:22,541 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38101, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 16:54:22,553 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39639] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:22,563 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39639] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:22,570 DEBUG [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738 2023-07-20 16:54:22,570 DEBUG [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738 2023-07-20 16:54:22,570 DEBUG [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:38055 2023-07-20 16:54:22,570 DEBUG [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:38055 2023-07-20 16:54:22,571 DEBUG [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 16:54:22,571 DEBUG [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 16:54:22,577 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 16:54:22,578 DEBUG [RS:1;jenkins-hbase4:34183] zookeeper.ZKUtil(162): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:22,578 DEBUG [RS:0;jenkins-hbase4:33103] zookeeper.ZKUtil(162): regionserver:33103-0x10183a7c1310001, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:22,578 WARN [RS:1;jenkins-hbase4:34183] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 16:54:22,578 WARN [RS:0;jenkins-hbase4:33103] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 16:54:22,579 INFO [RS:1;jenkins-hbase4:34183] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:54:22,580 INFO [RS:0;jenkins-hbase4:33103] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:54:22,580 INFO [RS:1;jenkins-hbase4:34183] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:54:22,580 INFO [RS:0;jenkins-hbase4:33103] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:54:22,581 DEBUG [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(1948): logDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:22,581 DEBUG [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(1948): logDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:22,581 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34183,1689872061357] 2023-07-20 16:54:22,581 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33103,1689872061305] 2023-07-20 16:54:22,591 DEBUG [RS:1;jenkins-hbase4:34183] zookeeper.ZKUtil(162): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:22,591 DEBUG [RS:0;jenkins-hbase4:33103] zookeeper.ZKUtil(162): regionserver:33103-0x10183a7c1310001, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:22,592 DEBUG [RS:1;jenkins-hbase4:34183] zookeeper.ZKUtil(162): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:22,592 DEBUG [RS:0;jenkins-hbase4:33103] zookeeper.ZKUtil(162): regionserver:33103-0x10183a7c1310001, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:22,602 DEBUG [RS:1;jenkins-hbase4:34183] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 16:54:22,602 DEBUG [RS:0;jenkins-hbase4:33103] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 16:54:22,611 INFO [RS:1;jenkins-hbase4:34183] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 16:54:22,611 INFO [RS:0;jenkins-hbase4:33103] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 16:54:22,631 INFO [RS:0;jenkins-hbase4:33103] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 16:54:22,631 INFO [RS:1;jenkins-hbase4:34183] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 16:54:22,635 INFO [RS:0;jenkins-hbase4:33103] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 16:54:22,635 INFO [RS:1;jenkins-hbase4:34183] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 16:54:22,636 INFO [RS:0;jenkins-hbase4:33103] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,636 INFO [RS:1;jenkins-hbase4:34183] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,637 INFO [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 16:54:22,637 INFO [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 16:54:22,645 INFO [RS:1;jenkins-hbase4:34183] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,645 INFO [RS:0;jenkins-hbase4:33103] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,645 DEBUG [RS:1;jenkins-hbase4:34183] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,645 DEBUG [RS:0;jenkins-hbase4:33103] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,645 DEBUG [RS:1;jenkins-hbase4:34183] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,645 DEBUG [RS:0;jenkins-hbase4:33103] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,645 DEBUG [RS:1;jenkins-hbase4:34183] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,646 DEBUG [RS:0;jenkins-hbase4:33103] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,646 DEBUG [RS:1;jenkins-hbase4:34183] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,646 DEBUG [RS:0;jenkins-hbase4:33103] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,646 DEBUG [RS:1;jenkins-hbase4:34183] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,646 DEBUG [RS:0;jenkins-hbase4:33103] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,646 DEBUG [RS:1;jenkins-hbase4:34183] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 16:54:22,646 DEBUG [RS:0;jenkins-hbase4:33103] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 16:54:22,646 DEBUG [RS:1;jenkins-hbase4:34183] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,646 DEBUG [RS:0;jenkins-hbase4:33103] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,646 DEBUG [RS:1;jenkins-hbase4:34183] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,646 DEBUG [RS:0;jenkins-hbase4:33103] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,646 DEBUG [RS:1;jenkins-hbase4:34183] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,647 DEBUG [RS:0;jenkins-hbase4:33103] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,647 DEBUG [RS:1;jenkins-hbase4:34183] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,647 DEBUG [RS:0;jenkins-hbase4:33103] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:22,648 INFO [RS:1;jenkins-hbase4:34183] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,648 INFO [RS:1;jenkins-hbase4:34183] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,648 INFO [RS:1;jenkins-hbase4:34183] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,651 INFO [RS:0;jenkins-hbase4:33103] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,651 INFO [RS:0;jenkins-hbase4:33103] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,651 INFO [RS:0;jenkins-hbase4:33103] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,667 INFO [RS:0;jenkins-hbase4:33103] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 16:54:22,667 INFO [RS:1;jenkins-hbase4:34183] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 16:54:22,670 INFO [RS:1;jenkins-hbase4:34183] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34183,1689872061357-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,670 INFO [RS:0;jenkins-hbase4:33103] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33103,1689872061305-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:22,682 DEBUG [jenkins-hbase4:39639] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=2, allServersCount=2 2023-07-20 16:54:22,686 DEBUG [jenkins-hbase4:39639] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 16:54:22,693 DEBUG [jenkins-hbase4:39639] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 16:54:22,693 DEBUG [jenkins-hbase4:39639] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 16:54:22,693 DEBUG [jenkins-hbase4:39639] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 16:54:22,699 INFO [RS:1;jenkins-hbase4:34183] regionserver.Replication(203): jenkins-hbase4.apache.org,34183,1689872061357 started 2023-07-20 16:54:22,699 INFO [RS:0;jenkins-hbase4:33103] regionserver.Replication(203): jenkins-hbase4.apache.org,33103,1689872061305 started 2023-07-20 16:54:22,699 INFO [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34183,1689872061357, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34183, sessionid=0x10183a7c1310002 2023-07-20 16:54:22,699 INFO [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,33103,1689872061305, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33103, sessionid=0x10183a7c1310001 2023-07-20 16:54:22,699 DEBUG [RS:1;jenkins-hbase4:34183] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 16:54:22,699 DEBUG [RS:0;jenkins-hbase4:33103] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 16:54:22,699 DEBUG [RS:1;jenkins-hbase4:34183] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:22,700 DEBUG [RS:0;jenkins-hbase4:33103] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:22,700 DEBUG [RS:1;jenkins-hbase4:34183] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34183,1689872061357' 2023-07-20 16:54:22,700 DEBUG [RS:0;jenkins-hbase4:33103] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33103,1689872061305' 2023-07-20 16:54:22,701 DEBUG [RS:1;jenkins-hbase4:34183] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-20 16:54:22,701 DEBUG [RS:0;jenkins-hbase4:33103] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-20 16:54:22,702 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34183,1689872061357, state=OPENING 2023-07-20 16:54:22,702 DEBUG [RS:0;jenkins-hbase4:33103] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-20 16:54:22,702 DEBUG [RS:1;jenkins-hbase4:34183] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-20 16:54:22,702 DEBUG [RS:0;jenkins-hbase4:33103] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 16:54:22,703 DEBUG [RS:1;jenkins-hbase4:34183] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 16:54:22,703 DEBUG [RS:1;jenkins-hbase4:34183] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 16:54:22,703 DEBUG [RS:0;jenkins-hbase4:33103] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 16:54:22,703 DEBUG [RS:1;jenkins-hbase4:34183] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:22,703 DEBUG [RS:0;jenkins-hbase4:33103] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:22,703 DEBUG [RS:0;jenkins-hbase4:33103] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33103,1689872061305' 2023-07-20 16:54:22,703 DEBUG [RS:1;jenkins-hbase4:34183] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34183,1689872061357' 2023-07-20 16:54:22,703 DEBUG [RS:1;jenkins-hbase4:34183] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-20 16:54:22,703 DEBUG [RS:0;jenkins-hbase4:33103] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-20 16:54:22,704 DEBUG [RS:1;jenkins-hbase4:34183] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-20 16:54:22,704 DEBUG [RS:0;jenkins-hbase4:33103] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-20 16:54:22,705 DEBUG [RS:1;jenkins-hbase4:34183] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 16:54:22,705 DEBUG [RS:0;jenkins-hbase4:33103] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 16:54:22,705 INFO [RS:1;jenkins-hbase4:34183] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 16:54:22,705 INFO [RS:0;jenkins-hbase4:33103] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 16:54:22,706 INFO [RS:1;jenkins-hbase4:34183] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 16:54:22,706 INFO [RS:0;jenkins-hbase4:33103] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 16:54:22,713 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-20 16:54:22,714 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:54:22,715 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-20 16:54:22,720 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34183,1689872061357}] 2023-07-20 16:54:22,814 INFO [RS:0;jenkins-hbase4:33103] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:22,814 INFO [RS:1;jenkins-hbase4:34183] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:22,818 INFO [RS:0;jenkins-hbase4:33103] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0, suffix=, logDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305, archiveDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs, maxLogs=10 2023-07-20 16:54:22,818 INFO [RS:1;jenkins-hbase4:34183] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0, suffix=, logDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357, archiveDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs, maxLogs=10 2023-07-20 16:54:22,875 INFO [RS:1;jenkins-hbase4:34183] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357/jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0.1689872062821 2023-07-20 16:54:22,875 INFO [RS:0;jenkins-hbase4:33103] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0.1689872062821 2023-07-20 16:54:22,875 DEBUG [RS:1;jenkins-hbase4:34183] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40699,DS-82a10023-4ecc-44e1-b86d-cd4749f69898,DISK], DatanodeInfoWithStorage[127.0.0.1:45027,DS-87ab81e2-6016-492e-a423-b308cc471afd,DISK]] 2023-07-20 16:54:22,876 DEBUG [RS:0;jenkins-hbase4:33103] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45027,DS-87ab81e2-6016-492e-a423-b308cc471afd,DISK], DatanodeInfoWithStorage[127.0.0.1:40699,DS-82a10023-4ecc-44e1-b86d-cd4749f69898,DISK]] 2023-07-20 16:54:22,908 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:22,910 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 16:54:22,914 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37880, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 16:54:22,929 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-20 16:54:22,930 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:54:22,930 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:54:22,930 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:22,934 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34183%2C1689872061357.meta, suffix=.meta, logDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357, archiveDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs, maxLogs=10 2023-07-20 16:54:22,957 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357/jenkins-hbase4.apache.org%2C34183%2C1689872061357.meta.1689872062936.meta 2023-07-20 16:54:22,957 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40699,DS-82a10023-4ecc-44e1-b86d-cd4749f69898,DISK], DatanodeInfoWithStorage[127.0.0.1:45027,DS-87ab81e2-6016-492e-a423-b308cc471afd,DISK]] 2023-07-20 16:54:22,958 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:22,959 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-20 16:54:22,976 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-20 16:54:22,982 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-20 16:54:22,988 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-20 16:54:22,988 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:22,988 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-20 16:54:22,988 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-20 16:54:22,991 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-20 16:54:22,993 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/info 2023-07-20 16:54:22,993 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/info 2023-07-20 16:54:22,994 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-20 16:54:22,995 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:22,995 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-20 16:54:22,996 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/rep_barrier 2023-07-20 16:54:22,996 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/rep_barrier 2023-07-20 16:54:22,997 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-20 16:54:22,998 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:22,998 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-20 16:54:23,002 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/table 2023-07-20 16:54:23,002 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/table 2023-07-20 16:54:23,003 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-20 16:54:23,004 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:23,005 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740 2023-07-20 16:54:23,008 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740 2023-07-20 16:54:23,012 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-20 16:54:23,014 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-20 16:54:23,016 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58891399, jitterRate=-0.12244977056980133}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-20 16:54:23,016 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-20 16:54:23,027 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689872062900 2023-07-20 16:54:23,044 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-20 16:54:23,045 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-20 16:54:23,046 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34183,1689872061357, state=OPEN 2023-07-20 16:54:23,050 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-20 16:54:23,050 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-20 16:54:23,055 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-20 16:54:23,055 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34183,1689872061357 in 331 msec 2023-07-20 16:54:23,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-20 16:54:23,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 549 msec 2023-07-20 16:54:23,068 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 811 msec 2023-07-20 16:54:23,068 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689872063068, completionTime=-1 2023-07-20 16:54:23,068 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=2; waited=0ms, expected min=2 server(s), max=2 server(s), master is running 2023-07-20 16:54:23,069 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-20 16:54:23,138 DEBUG [hconnection-0x3a5b1ec4-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:23,140 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37886, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:54:23,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=2 2023-07-20 16:54:23,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689872123162 2023-07-20 16:54:23,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689872183162 2023-07-20 16:54:23,163 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 93 msec 2023-07-20 16:54:23,185 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39639,1689872060057-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:23,185 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39639,1689872060057-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:23,185 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39639,1689872060057-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:23,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:39639, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:23,188 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:23,193 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-20 16:54:23,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-20 16:54:23,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-20 16:54:23,213 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-20 16:54:23,215 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-20 16:54:23,217 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-20 16:54:23,240 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp/data/hbase/namespace/eec95f84b91ff86943540982735963f6 2023-07-20 16:54:23,243 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp/data/hbase/namespace/eec95f84b91ff86943540982735963f6 empty. 2023-07-20 16:54:23,244 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp/data/hbase/namespace/eec95f84b91ff86943540982735963f6 2023-07-20 16:54:23,244 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-20 16:54:23,276 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-20 16:54:23,278 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => eec95f84b91ff86943540982735963f6, NAME => 'hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp 2023-07-20 16:54:23,300 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:23,300 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing eec95f84b91ff86943540982735963f6, disabling compactions & flushes 2023-07-20 16:54:23,300 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:54:23,300 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:54:23,300 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. after waiting 0 ms 2023-07-20 16:54:23,300 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:54:23,300 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:54:23,300 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for eec95f84b91ff86943540982735963f6: 2023-07-20 16:54:23,305 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-20 16:54:23,322 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689872063309"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689872063309"}]},"ts":"1689872063309"} 2023-07-20 16:54:23,349 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-20 16:54:23,351 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-20 16:54:23,356 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689872063352"}]},"ts":"1689872063352"} 2023-07-20 16:54:23,362 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-20 16:54:23,367 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 16:54:23,369 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 16:54:23,369 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 16:54:23,369 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 16:54:23,372 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=eec95f84b91ff86943540982735963f6, ASSIGN}] 2023-07-20 16:54:23,374 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=eec95f84b91ff86943540982735963f6, ASSIGN 2023-07-20 16:54:23,376 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=eec95f84b91ff86943540982735963f6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34183,1689872061357; forceNewPlan=false, retain=false 2023-07-20 16:54:23,528 INFO [jenkins-hbase4:39639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-20 16:54:23,529 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=eec95f84b91ff86943540982735963f6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:23,530 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689872063529"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872063529"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872063529"}]},"ts":"1689872063529"} 2023-07-20 16:54:23,536 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure eec95f84b91ff86943540982735963f6, server=jenkins-hbase4.apache.org,34183,1689872061357}] 2023-07-20 16:54:23,697 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:54:23,698 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:23,701 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1, suffix=, logDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357, archiveDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs, maxLogs=10 2023-07-20 16:54:23,714 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357/jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1.1689872063702 2023-07-20 16:54:23,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45027,DS-87ab81e2-6016-492e-a423-b308cc471afd,DISK], DatanodeInfoWithStorage[127.0.0.1:40699,DS-82a10023-4ecc-44e1-b86d-cd4749f69898,DISK]] 2023-07-20 16:54:23,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => eec95f84b91ff86943540982735963f6, NAME => 'hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:23,715 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace eec95f84b91ff86943540982735963f6 2023-07-20 16:54:23,715 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:23,715 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for eec95f84b91ff86943540982735963f6 2023-07-20 16:54:23,715 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for eec95f84b91ff86943540982735963f6 2023-07-20 16:54:23,717 INFO [StoreOpener-eec95f84b91ff86943540982735963f6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region eec95f84b91ff86943540982735963f6 2023-07-20 16:54:23,720 DEBUG [StoreOpener-eec95f84b91ff86943540982735963f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6/info 2023-07-20 16:54:23,720 DEBUG [StoreOpener-eec95f84b91ff86943540982735963f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6/info 2023-07-20 16:54:23,720 INFO [StoreOpener-eec95f84b91ff86943540982735963f6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eec95f84b91ff86943540982735963f6 columnFamilyName info 2023-07-20 16:54:23,722 INFO [StoreOpener-eec95f84b91ff86943540982735963f6-1] regionserver.HStore(310): Store=eec95f84b91ff86943540982735963f6/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:23,724 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6 2023-07-20 16:54:23,725 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6 2023-07-20 16:54:23,729 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for eec95f84b91ff86943540982735963f6 2023-07-20 16:54:23,733 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 16:54:23,734 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened eec95f84b91ff86943540982735963f6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58830648, jitterRate=-0.12335503101348877}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 16:54:23,734 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for eec95f84b91ff86943540982735963f6: 2023-07-20 16:54:23,736 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6., pid=6, masterSystemTime=1689872063690 2023-07-20 16:54:23,741 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:54:23,741 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:54:23,743 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=eec95f84b91ff86943540982735963f6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:23,743 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689872063743"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872063743"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872063743"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872063743"}]},"ts":"1689872063743"} 2023-07-20 16:54:23,751 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-20 16:54:23,752 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure eec95f84b91ff86943540982735963f6, server=jenkins-hbase4.apache.org,34183,1689872061357 in 211 msec 2023-07-20 16:54:23,755 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-20 16:54:23,756 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=eec95f84b91ff86943540982735963f6, ASSIGN in 380 msec 2023-07-20 16:54:23,757 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-20 16:54:23,757 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689872063757"}]},"ts":"1689872063757"} 2023-07-20 16:54:23,762 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-20 16:54:23,766 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-20 16:54:23,768 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 562 msec 2023-07-20 16:54:23,815 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/namespace 2023-07-20 16:54:23,817 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/namespace 2023-07-20 16:54:23,817 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:54:23,857 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-20 16:54:23,884 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-20 16:54:23,892 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 45 msec 2023-07-20 16:54:23,909 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-20 16:54:23,922 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/namespace 2023-07-20 16:54:23,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 18 msec 2023-07-20 16:54:23,937 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-20 16:54:23,939 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-20 16:54:23,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.509sec 2023-07-20 16:54:23,942 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-20 16:54:23,944 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-20 16:54:23,944 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-20 16:54:23,946 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39639,1689872060057-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-20 16:54:23,946 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39639,1689872060057-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-20 16:54:23,955 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-20 16:54:23,997 DEBUG [Listener at localhost/36209] zookeeper.ReadOnlyZKClient(139): Connect 0x3448932c to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:24,003 DEBUG [Listener at localhost/36209] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@107bcd57, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:24,016 DEBUG [hconnection-0x120946e1-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:24,028 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37892, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:54:24,037 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,39639,1689872060057 2023-07-20 16:54:24,038 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=4, rsPorts=, rsClass=null, numDataNodes=4, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-20 16:54:24,038 INFO [Listener at localhost/36209] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/cluster_b91dc339-012d-6b0e-9c91-88d86ff52b70, deleteOnExit=true 2023-07-20 16:54:24,038 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-20 16:54:24,049 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/test.cache.data in system properties and HBase conf 2023-07-20 16:54:24,049 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/hadoop.tmp.dir in system properties and HBase conf 2023-07-20 16:54:24,049 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/hadoop.log.dir in system properties and HBase conf 2023-07-20 16:54:24,049 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-20 16:54:24,050 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-20 16:54:24,050 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-20 16:54:24,050 DEBUG [Listener at localhost/36209] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-20 16:54:24,050 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-20 16:54:24,050 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-20 16:54:24,051 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-20 16:54:24,051 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-20 16:54:24,051 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-20 16:54:24,051 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-20 16:54:24,051 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-20 16:54:24,051 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-20 16:54:24,051 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-20 16:54:24,051 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/nfs.dump.dir in system properties and HBase conf 2023-07-20 16:54:24,051 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/java.io.tmpdir in system properties and HBase conf 2023-07-20 16:54:24,052 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-20 16:54:24,052 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-20 16:54:24,052 INFO [Listener at localhost/36209] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-20 16:54:24,056 WARN [Listener at localhost/36209] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-20 16:54:24,057 WARN [Listener at localhost/36209] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-20 16:54:24,103 WARN [Listener at localhost/36209] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 16:54:24,105 INFO [Listener at localhost/36209] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 16:54:24,112 INFO [Listener at localhost/36209] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/java.io.tmpdir/Jetty_localhost_33883_hdfs____.dhjngj/webapp 2023-07-20 16:54:24,207 INFO [Listener at localhost/36209] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33883 2023-07-20 16:54:24,216 WARN [Listener at localhost/36209] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-20 16:54:24,216 WARN [Listener at localhost/36209] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-20 16:54:24,326 WARN [Listener at localhost/33767] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 16:54:24,347 WARN [Listener at localhost/33767] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-20 16:54:24,350 WARN [Listener at localhost/33767] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 16:54:24,352 INFO [Listener at localhost/33767] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 16:54:24,357 INFO [Listener at localhost/33767] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/java.io.tmpdir/Jetty_localhost_43101_datanode____7w0tiy/webapp 2023-07-20 16:54:24,450 INFO [Listener at localhost/33767] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43101 2023-07-20 16:54:24,461 WARN [Listener at localhost/36205] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 16:54:24,489 WARN [Listener at localhost/36205] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-20 16:54:24,492 WARN [Listener at localhost/36205] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 16:54:24,493 INFO [Listener at localhost/36205] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 16:54:24,499 INFO [Listener at localhost/36205] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/java.io.tmpdir/Jetty_localhost_38463_datanode____t8xt8z/webapp 2023-07-20 16:54:24,599 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb9f2653cec06a846: Processing first storage report for DS-69ae4973-7589-41c7-bb37-0af0ca263f5e from datanode 5c54da54-6aae-49ea-ad40-5417a6f96449 2023-07-20 16:54:24,600 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb9f2653cec06a846: from storage DS-69ae4973-7589-41c7-bb37-0af0ca263f5e node DatanodeRegistration(127.0.0.1:36101, datanodeUuid=5c54da54-6aae-49ea-ad40-5417a6f96449, infoPort=35253, infoSecurePort=0, ipcPort=36205, storageInfo=lv=-57;cid=testClusterID;nsid=1066529364;c=1689872064059), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-20 16:54:24,600 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb9f2653cec06a846: Processing first storage report for DS-da6c4912-726c-437d-94c9-ee38e9a7e95a from datanode 5c54da54-6aae-49ea-ad40-5417a6f96449 2023-07-20 16:54:24,600 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb9f2653cec06a846: from storage DS-da6c4912-726c-437d-94c9-ee38e9a7e95a node DatanodeRegistration(127.0.0.1:36101, datanodeUuid=5c54da54-6aae-49ea-ad40-5417a6f96449, infoPort=35253, infoSecurePort=0, ipcPort=36205, storageInfo=lv=-57;cid=testClusterID;nsid=1066529364;c=1689872064059), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 16:54:24,625 INFO [Listener at localhost/36205] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38463 2023-07-20 16:54:24,636 WARN [Listener at localhost/37639] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 16:54:24,661 WARN [Listener at localhost/37639] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-20 16:54:24,664 WARN [Listener at localhost/37639] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 16:54:24,665 INFO [Listener at localhost/37639] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 16:54:24,671 INFO [Listener at localhost/37639] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/java.io.tmpdir/Jetty_localhost_38201_datanode____.vrxtyj/webapp 2023-07-20 16:54:24,772 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x748db4d0faa0cef6: Processing first storage report for DS-f901468c-90d8-4877-8572-70b06cbb96ab from datanode 03416ff6-ea90-4b13-9f8c-c5bf8a981c49 2023-07-20 16:54:24,772 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x748db4d0faa0cef6: from storage DS-f901468c-90d8-4877-8572-70b06cbb96ab node DatanodeRegistration(127.0.0.1:38315, datanodeUuid=03416ff6-ea90-4b13-9f8c-c5bf8a981c49, infoPort=45793, infoSecurePort=0, ipcPort=37639, storageInfo=lv=-57;cid=testClusterID;nsid=1066529364;c=1689872064059), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 16:54:24,772 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x748db4d0faa0cef6: Processing first storage report for DS-a9004737-84de-4983-ba47-6ee202b670f5 from datanode 03416ff6-ea90-4b13-9f8c-c5bf8a981c49 2023-07-20 16:54:24,772 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x748db4d0faa0cef6: from storage DS-a9004737-84de-4983-ba47-6ee202b670f5 node DatanodeRegistration(127.0.0.1:38315, datanodeUuid=03416ff6-ea90-4b13-9f8c-c5bf8a981c49, infoPort=45793, infoSecurePort=0, ipcPort=37639, storageInfo=lv=-57;cid=testClusterID;nsid=1066529364;c=1689872064059), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 16:54:24,789 INFO [Listener at localhost/37639] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38201 2023-07-20 16:54:24,798 WARN [Listener at localhost/34417] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 16:54:24,820 WARN [Listener at localhost/34417] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-20 16:54:24,823 WARN [Listener at localhost/34417] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-20 16:54:24,824 INFO [Listener at localhost/34417] log.Slf4jLog(67): jetty-6.1.26 2023-07-20 16:54:24,831 INFO [Listener at localhost/34417] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/java.io.tmpdir/Jetty_localhost_37711_datanode____lxp3bu/webapp 2023-07-20 16:54:24,925 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3fae9d4635c4d989: Processing first storage report for DS-21386278-ed61-414b-9d04-87c474a58f9b from datanode 255edda2-8292-4de8-8e52-18ab644e2dec 2023-07-20 16:54:24,926 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3fae9d4635c4d989: from storage DS-21386278-ed61-414b-9d04-87c474a58f9b node DatanodeRegistration(127.0.0.1:35885, datanodeUuid=255edda2-8292-4de8-8e52-18ab644e2dec, infoPort=45359, infoSecurePort=0, ipcPort=34417, storageInfo=lv=-57;cid=testClusterID;nsid=1066529364;c=1689872064059), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 16:54:24,926 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3fae9d4635c4d989: Processing first storage report for DS-a59c1cb0-6594-4919-a336-68db5a31a657 from datanode 255edda2-8292-4de8-8e52-18ab644e2dec 2023-07-20 16:54:24,926 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3fae9d4635c4d989: from storage DS-a59c1cb0-6594-4919-a336-68db5a31a657 node DatanodeRegistration(127.0.0.1:35885, datanodeUuid=255edda2-8292-4de8-8e52-18ab644e2dec, infoPort=45359, infoSecurePort=0, ipcPort=34417, storageInfo=lv=-57;cid=testClusterID;nsid=1066529364;c=1689872064059), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 16:54:24,939 INFO [Listener at localhost/34417] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37711 2023-07-20 16:54:24,949 WARN [Listener at localhost/33489] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-20 16:54:25,056 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xbbb520e27875554a: Processing first storage report for DS-e3134465-ba87-4999-ba03-bbcb207f3c3b from datanode 1b5916d6-dbef-485a-8fd4-49d0508d7da0 2023-07-20 16:54:25,056 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xbbb520e27875554a: from storage DS-e3134465-ba87-4999-ba03-bbcb207f3c3b node DatanodeRegistration(127.0.0.1:35847, datanodeUuid=1b5916d6-dbef-485a-8fd4-49d0508d7da0, infoPort=42017, infoSecurePort=0, ipcPort=33489, storageInfo=lv=-57;cid=testClusterID;nsid=1066529364;c=1689872064059), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 16:54:25,056 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xbbb520e27875554a: Processing first storage report for DS-28cd4fb9-f6b1-4783-912c-4a78f8c8b408 from datanode 1b5916d6-dbef-485a-8fd4-49d0508d7da0 2023-07-20 16:54:25,056 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xbbb520e27875554a: from storage DS-28cd4fb9-f6b1-4783-912c-4a78f8c8b408 node DatanodeRegistration(127.0.0.1:35847, datanodeUuid=1b5916d6-dbef-485a-8fd4-49d0508d7da0, infoPort=42017, infoSecurePort=0, ipcPort=33489, storageInfo=lv=-57;cid=testClusterID;nsid=1066529364;c=1689872064059), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-20 16:54:25,060 DEBUG [Listener at localhost/33489] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503 2023-07-20 16:54:25,060 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:25,061 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:25,103 INFO [Listener at localhost/33489] util.FSUtils(471): Created version file at hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a with version=8 2023-07-20 16:54:25,104 INFO [Listener at localhost/33489] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/hbase-staging 2023-07-20 16:54:25,107 INFO [Listener at localhost/33489] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 16:54:25,107 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,107 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,107 INFO [Listener at localhost/33489] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 16:54:25,107 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,107 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 16:54:25,108 INFO [Listener at localhost/33489] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 16:54:25,109 INFO [Listener at localhost/33489] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45867 2023-07-20 16:54:25,110 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:25,112 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:25,113 INFO [Listener at localhost/33489] zookeeper.RecoverableZooKeeper(93): Process identifier=master:45867 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:25,117 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:458670x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:25,118 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:45867-0x10183a7c1310007 connected 2023-07-20 16:54:25,131 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 16:54:25,132 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:54:25,132 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 16:54:25,133 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45867 2023-07-20 16:54:25,133 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45867 2023-07-20 16:54:25,133 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45867 2023-07-20 16:54:25,133 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45867 2023-07-20 16:54:25,134 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45867 2023-07-20 16:54:25,134 INFO [Listener at localhost/33489] master.HMaster(444): hbase.rootdir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a, hbase.cluster.distributed=false 2023-07-20 16:54:25,149 INFO [Listener at localhost/33489] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 16:54:25,149 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,149 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,149 INFO [Listener at localhost/33489] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 16:54:25,149 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,149 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 16:54:25,149 INFO [Listener at localhost/33489] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 16:54:25,151 INFO [Listener at localhost/33489] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45493 2023-07-20 16:54:25,152 INFO [Listener at localhost/33489] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 16:54:25,153 DEBUG [Listener at localhost/33489] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 16:54:25,153 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:25,154 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:25,155 INFO [Listener at localhost/33489] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45493 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:25,159 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:454930x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:25,160 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:454930x0, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 16:54:25,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45493-0x10183a7c1310008 connected 2023-07-20 16:54:25,161 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:54:25,162 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 16:54:25,165 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45493 2023-07-20 16:54:25,166 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45493 2023-07-20 16:54:25,167 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45493 2023-07-20 16:54:25,167 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45493 2023-07-20 16:54:25,167 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45493 2023-07-20 16:54:25,179 INFO [Listener at localhost/33489] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 16:54:25,179 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,179 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,179 INFO [Listener at localhost/33489] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 16:54:25,179 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,179 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 16:54:25,179 INFO [Listener at localhost/33489] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 16:54:25,182 INFO [Listener at localhost/33489] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46685 2023-07-20 16:54:25,182 INFO [Listener at localhost/33489] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 16:54:25,184 DEBUG [Listener at localhost/33489] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 16:54:25,184 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:25,185 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:25,186 INFO [Listener at localhost/33489] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46685 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:25,191 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:466850x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:25,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46685-0x10183a7c1310009 connected 2023-07-20 16:54:25,192 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 16:54:25,193 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:54:25,193 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 16:54:25,194 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46685 2023-07-20 16:54:25,194 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46685 2023-07-20 16:54:25,194 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46685 2023-07-20 16:54:25,195 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46685 2023-07-20 16:54:25,196 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46685 2023-07-20 16:54:25,208 INFO [Listener at localhost/33489] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 16:54:25,208 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,208 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,208 INFO [Listener at localhost/33489] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 16:54:25,208 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,208 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 16:54:25,208 INFO [Listener at localhost/33489] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 16:54:25,210 INFO [Listener at localhost/33489] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43901 2023-07-20 16:54:25,211 INFO [Listener at localhost/33489] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 16:54:25,212 DEBUG [Listener at localhost/33489] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 16:54:25,212 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:25,213 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:25,214 INFO [Listener at localhost/33489] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43901 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:25,219 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:439010x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:25,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:43901-0x10183a7c131000a connected 2023-07-20 16:54:25,221 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 16:54:25,222 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:54:25,222 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 16:54:25,225 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43901 2023-07-20 16:54:25,225 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43901 2023-07-20 16:54:25,226 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43901 2023-07-20 16:54:25,227 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43901 2023-07-20 16:54:25,227 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43901 2023-07-20 16:54:25,239 INFO [Listener at localhost/33489] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 16:54:25,239 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,239 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,239 INFO [Listener at localhost/33489] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 16:54:25,239 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:25,239 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 16:54:25,239 INFO [Listener at localhost/33489] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 16:54:25,241 INFO [Listener at localhost/33489] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45821 2023-07-20 16:54:25,242 INFO [Listener at localhost/33489] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 16:54:25,243 DEBUG [Listener at localhost/33489] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 16:54:25,243 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:25,244 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:25,245 INFO [Listener at localhost/33489] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45821 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:25,249 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:458210x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:25,250 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45821-0x10183a7c131000b connected 2023-07-20 16:54:25,250 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 16:54:25,251 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:54:25,251 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 16:54:25,255 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45821 2023-07-20 16:54:25,258 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45821 2023-07-20 16:54:25,259 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45821 2023-07-20 16:54:25,259 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45821 2023-07-20 16:54:25,261 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45821 2023-07-20 16:54:25,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,45867,1689872065106 2023-07-20 16:54:25,264 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-20 16:54:25,265 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,45867,1689872065106 2023-07-20 16:54:25,267 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 16:54:25,267 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 16:54:25,267 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 16:54:25,267 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 16:54:25,267 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 16:54:25,267 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:54:25,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-20 16:54:25,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-20 16:54:25,273 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,45867,1689872065106 from backup master directory 2023-07-20 16:54:25,275 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,45867,1689872065106 2023-07-20 16:54:25,275 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-20 16:54:25,276 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 16:54:25,276 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,45867,1689872065106 2023-07-20 16:54:25,310 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/hbase.id with ID: de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:54:25,328 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:25,331 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:54:25,353 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x7c50b214 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:25,358 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@63d19b11, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:25,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 16:54:25,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-20 16:54:25,359 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:54:25,359 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:54:25,361 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store-tmp 2023-07-20 16:54:25,385 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:25,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-20 16:54:25,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:25,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:25,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-20 16:54:25,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:25,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:25,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 16:54:25,387 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,45867,1689872065106 2023-07-20 16:54:25,388 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:25,391 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45867%2C1689872065106.jenkins-hbase4.apache.org%2C45867%2C1689872065106.regiongroup-0, suffix=, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,45867,1689872065106, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/oldWALs, maxLogs=10 2023-07-20 16:54:25,408 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,45867,1689872065106/jenkins-hbase4.apache.org%2C45867%2C1689872065106.jenkins-hbase4.apache.org%2C45867%2C1689872065106.regiongroup-0.1689872065391 2023-07-20 16:54:25,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35885,DS-21386278-ed61-414b-9d04-87c474a58f9b,DISK], DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK], DatanodeInfoWithStorage[127.0.0.1:35847,DS-e3134465-ba87-4999-ba03-bbcb207f3c3b,DISK]] 2023-07-20 16:54:25,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:25,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:25,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:25,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:25,412 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:25,414 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-20 16:54:25,415 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-20 16:54:25,415 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:25,417 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:25,418 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:25,423 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:25,426 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 16:54:25,427 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59667449, jitterRate=-0.11088572442531586}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 16:54:25,427 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 16:54:25,427 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-20 16:54:25,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-20 16:54:25,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-20 16:54:25,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-20 16:54:25,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-20 16:54:25,431 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 1 msec 2023-07-20 16:54:25,431 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-20 16:54:25,433 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-20 16:54:25,434 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-20 16:54:25,445 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-20 16:54:25,445 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-20 16:54:25,446 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-20 16:54:25,446 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-20 16:54:25,447 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-20 16:54:25,449 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:54:25,451 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-20 16:54:25,451 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-20 16:54:25,452 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-20 16:54:25,454 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 16:54:25,454 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 16:54:25,454 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 16:54:25,454 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:54:25,454 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 16:54:25,454 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 16:54:25,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,45867,1689872065106, sessionid=0x10183a7c1310007, setting cluster-up flag (Was=false) 2023-07-20 16:54:25,461 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:54:25,470 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-20 16:54:25,471 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45867,1689872065106 2023-07-20 16:54:25,475 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:54:25,485 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-20 16:54:25,486 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45867,1689872065106 2023-07-20 16:54:25,487 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.hbase-snapshot/.tmp 2023-07-20 16:54:25,491 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-20 16:54:25,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:54:25,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:54:25,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:54:25,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:54:25,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-20 16:54:25,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 16:54:25,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689872095494 2023-07-20 16:54:25,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-20 16:54:25,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-20 16:54:25,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-20 16:54:25,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-20 16:54:25,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-20 16:54:25,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-20 16:54:25,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-20 16:54:25,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-20 16:54:25,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-20 16:54:25,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-20 16:54:25,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-20 16:54:25,502 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872065500,5,FailOnTimeoutGroup] 2023-07-20 16:54:25,503 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872065502,5,FailOnTimeoutGroup] 2023-07-20 16:54:25,503 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,503 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-20 16:54:25,503 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-20 16:54:25,503 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,503 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-20 16:54:25,503 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,505 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-20 16:54:25,534 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-20 16:54:25,535 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-20 16:54:25,535 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a 2023-07-20 16:54:25,559 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:25,561 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-20 16:54:25,563 INFO [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(951): ClusterId : de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:54:25,563 INFO [RS:1;jenkins-hbase4:46685] regionserver.HRegionServer(951): ClusterId : de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:54:25,565 INFO [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(951): ClusterId : de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:54:25,566 DEBUG [RS:1;jenkins-hbase4:46685] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 16:54:25,567 DEBUG [RS:2;jenkins-hbase4:43901] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 16:54:25,564 DEBUG [RS:0;jenkins-hbase4:45493] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 16:54:25,568 INFO [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(951): ClusterId : de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:54:25,569 DEBUG [RS:3;jenkins-hbase4:45821] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 16:54:25,569 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info 2023-07-20 16:54:25,569 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-20 16:54:25,570 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:25,570 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-20 16:54:25,572 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/rep_barrier 2023-07-20 16:54:25,573 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-20 16:54:25,574 DEBUG [RS:1;jenkins-hbase4:46685] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 16:54:25,574 DEBUG [RS:1;jenkins-hbase4:46685] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 16:54:25,575 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:25,575 DEBUG [RS:3;jenkins-hbase4:45821] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 16:54:25,575 DEBUG [RS:0;jenkins-hbase4:45493] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 16:54:25,576 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-20 16:54:25,575 DEBUG [RS:3;jenkins-hbase4:45821] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 16:54:25,575 DEBUG [RS:2;jenkins-hbase4:43901] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 16:54:25,576 DEBUG [RS:2;jenkins-hbase4:43901] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 16:54:25,576 DEBUG [RS:0;jenkins-hbase4:45493] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 16:54:25,579 DEBUG [RS:2;jenkins-hbase4:43901] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 16:54:25,581 DEBUG [RS:2;jenkins-hbase4:43901] zookeeper.ReadOnlyZKClient(139): Connect 0x0cd00850 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:25,583 DEBUG [RS:0;jenkins-hbase4:45493] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 16:54:25,584 DEBUG [RS:3;jenkins-hbase4:45821] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 16:54:25,584 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/table 2023-07-20 16:54:25,584 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-20 16:54:25,585 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:25,590 DEBUG [RS:0;jenkins-hbase4:45493] zookeeper.ReadOnlyZKClient(139): Connect 0x7c6aa926 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:25,593 DEBUG [RS:3;jenkins-hbase4:45821] zookeeper.ReadOnlyZKClient(139): Connect 0x62bfb661 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:25,594 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740 2023-07-20 16:54:25,594 DEBUG [RS:1;jenkins-hbase4:46685] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 16:54:25,594 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740 2023-07-20 16:54:25,599 DEBUG [RS:2;jenkins-hbase4:43901] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@217f7ba, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:25,600 DEBUG [RS:2;jenkins-hbase4:43901] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6286320, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 16:54:25,601 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-20 16:54:25,603 DEBUG [RS:1;jenkins-hbase4:46685] zookeeper.ReadOnlyZKClient(139): Connect 0x17376923 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:25,605 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-20 16:54:25,615 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 16:54:25,617 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62873212, jitterRate=-0.06311613321304321}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-20 16:54:25,617 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-20 16:54:25,617 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-20 16:54:25,617 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-20 16:54:25,617 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-20 16:54:25,617 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-20 16:54:25,617 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-20 16:54:25,621 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-20 16:54:25,622 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-20 16:54:25,624 DEBUG [RS:3;jenkins-hbase4:45821] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@66802eee, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:25,624 DEBUG [RS:3;jenkins-hbase4:45821] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@d784901, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 16:54:25,629 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-20 16:54:25,629 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-20 16:54:25,629 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-20 16:54:25,630 DEBUG [RS:2;jenkins-hbase4:43901] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:43901 2023-07-20 16:54:25,630 INFO [RS:2;jenkins-hbase4:43901] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 16:54:25,630 INFO [RS:2;jenkins-hbase4:43901] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 16:54:25,630 DEBUG [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 16:54:25,633 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-20 16:54:25,635 DEBUG [RS:0;jenkins-hbase4:45493] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@72ad48db, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:25,635 DEBUG [RS:0;jenkins-hbase4:45493] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7692c256, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 16:54:25,635 INFO [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45867,1689872065106 with isa=jenkins-hbase4.apache.org/172.31.14.131:43901, startcode=1689872065207 2023-07-20 16:54:25,636 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-20 16:54:25,636 DEBUG [RS:2;jenkins-hbase4:43901] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 16:54:25,638 DEBUG [RS:3;jenkins-hbase4:45821] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase4:45821 2023-07-20 16:54:25,639 INFO [RS:3;jenkins-hbase4:45821] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 16:54:25,639 INFO [RS:3;jenkins-hbase4:45821] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 16:54:25,639 DEBUG [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 16:54:25,639 DEBUG [RS:1;jenkins-hbase4:46685] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@19f085b5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:25,639 DEBUG [RS:1;jenkins-hbase4:46685] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1dcf0253, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 16:54:25,639 INFO [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45867,1689872065106 with isa=jenkins-hbase4.apache.org/172.31.14.131:45821, startcode=1689872065238 2023-07-20 16:54:25,640 DEBUG [RS:3;jenkins-hbase4:45821] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 16:54:25,649 DEBUG [RS:1;jenkins-hbase4:46685] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:46685 2023-07-20 16:54:25,649 DEBUG [RS:0;jenkins-hbase4:45493] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:45493 2023-07-20 16:54:25,649 INFO [RS:1;jenkins-hbase4:46685] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 16:54:25,650 INFO [RS:0;jenkins-hbase4:45493] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 16:54:25,650 INFO [RS:0;jenkins-hbase4:45493] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 16:54:25,650 DEBUG [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 16:54:25,650 INFO [RS:1;jenkins-hbase4:46685] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 16:54:25,650 DEBUG [RS:1;jenkins-hbase4:46685] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 16:54:25,654 INFO [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45867,1689872065106 with isa=jenkins-hbase4.apache.org/172.31.14.131:45493, startcode=1689872065148 2023-07-20 16:54:25,654 INFO [RS:1;jenkins-hbase4:46685] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45867,1689872065106 with isa=jenkins-hbase4.apache.org/172.31.14.131:46685, startcode=1689872065178 2023-07-20 16:54:25,655 DEBUG [RS:0;jenkins-hbase4:45493] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 16:54:25,655 DEBUG [RS:1;jenkins-hbase4:46685] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 16:54:25,655 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56883, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 16:54:25,656 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34395, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 16:54:25,660 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45867] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:25,660 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52395, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 16:54:25,660 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50655, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 16:54:25,660 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45867] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:25,661 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45867] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:25,661 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45867] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:25,661 DEBUG [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a 2023-07-20 16:54:25,661 DEBUG [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33767 2023-07-20 16:54:25,661 DEBUG [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 16:54:25,662 DEBUG [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a 2023-07-20 16:54:25,662 DEBUG [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33767 2023-07-20 16:54:25,662 DEBUG [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 16:54:25,663 DEBUG [RS:1;jenkins-hbase4:46685] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a 2023-07-20 16:54:25,663 DEBUG [RS:1;jenkins-hbase4:46685] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33767 2023-07-20 16:54:25,663 DEBUG [RS:1;jenkins-hbase4:46685] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 16:54:25,664 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:54:25,664 DEBUG [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a 2023-07-20 16:54:25,664 DEBUG [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33767 2023-07-20 16:54:25,664 DEBUG [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 16:54:25,667 DEBUG [RS:2;jenkins-hbase4:43901] zookeeper.ZKUtil(162): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:25,668 WARN [RS:2;jenkins-hbase4:43901] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 16:54:25,668 INFO [RS:2;jenkins-hbase4:43901] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:54:25,668 INFO [RS:2;jenkins-hbase4:43901] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:54:25,668 DEBUG [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:25,668 DEBUG [RS:0;jenkins-hbase4:45493] zookeeper.ZKUtil(162): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:25,668 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45493,1689872065148] 2023-07-20 16:54:25,668 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43901,1689872065207] 2023-07-20 16:54:25,668 DEBUG [RS:1;jenkins-hbase4:46685] zookeeper.ZKUtil(162): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:25,669 DEBUG [RS:3;jenkins-hbase4:45821] zookeeper.ZKUtil(162): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:25,669 WARN [RS:3;jenkins-hbase4:45821] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 16:54:25,669 INFO [RS:3;jenkins-hbase4:45821] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:54:25,669 INFO [RS:3;jenkins-hbase4:45821] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:54:25,668 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45821,1689872065238] 2023-07-20 16:54:25,668 WARN [RS:0;jenkins-hbase4:45493] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 16:54:25,670 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46685,1689872065178] 2023-07-20 16:54:25,670 DEBUG [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:25,669 WARN [RS:1;jenkins-hbase4:46685] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 16:54:25,670 INFO [RS:0;jenkins-hbase4:45493] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:54:25,671 INFO [RS:1;jenkins-hbase4:46685] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:54:25,671 INFO [RS:0;jenkins-hbase4:45493] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:54:25,671 DEBUG [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:25,671 INFO [RS:1;jenkins-hbase4:46685] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:54:25,671 DEBUG [RS:1;jenkins-hbase4:46685] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:25,682 DEBUG [RS:2;jenkins-hbase4:43901] zookeeper.ZKUtil(162): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:25,683 DEBUG [RS:2;jenkins-hbase4:43901] zookeeper.ZKUtil(162): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:25,684 DEBUG [RS:3;jenkins-hbase4:45821] zookeeper.ZKUtil(162): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:25,684 DEBUG [RS:1;jenkins-hbase4:46685] zookeeper.ZKUtil(162): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:25,684 DEBUG [RS:0;jenkins-hbase4:45493] zookeeper.ZKUtil(162): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:25,684 DEBUG [RS:3;jenkins-hbase4:45821] zookeeper.ZKUtil(162): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:25,684 DEBUG [RS:2;jenkins-hbase4:43901] zookeeper.ZKUtil(162): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:25,685 DEBUG [RS:1;jenkins-hbase4:46685] zookeeper.ZKUtil(162): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:25,685 DEBUG [RS:3;jenkins-hbase4:45821] zookeeper.ZKUtil(162): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:25,685 DEBUG [RS:0;jenkins-hbase4:45493] zookeeper.ZKUtil(162): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:25,685 DEBUG [RS:2;jenkins-hbase4:43901] zookeeper.ZKUtil(162): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:25,685 DEBUG [RS:1;jenkins-hbase4:46685] zookeeper.ZKUtil(162): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:25,685 DEBUG [RS:3;jenkins-hbase4:45821] zookeeper.ZKUtil(162): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:25,685 DEBUG [RS:0;jenkins-hbase4:45493] zookeeper.ZKUtil(162): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:25,686 DEBUG [RS:1;jenkins-hbase4:46685] zookeeper.ZKUtil(162): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:25,686 DEBUG [RS:0;jenkins-hbase4:45493] zookeeper.ZKUtil(162): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:25,687 DEBUG [RS:2;jenkins-hbase4:43901] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 16:54:25,687 DEBUG [RS:3;jenkins-hbase4:45821] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 16:54:25,687 DEBUG [RS:1;jenkins-hbase4:46685] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 16:54:25,687 INFO [RS:2;jenkins-hbase4:43901] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 16:54:25,687 INFO [RS:3;jenkins-hbase4:45821] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 16:54:25,687 INFO [RS:1;jenkins-hbase4:46685] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 16:54:25,687 DEBUG [RS:0;jenkins-hbase4:45493] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 16:54:25,690 INFO [RS:0;jenkins-hbase4:45493] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 16:54:25,690 INFO [RS:2;jenkins-hbase4:43901] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 16:54:25,690 INFO [RS:2;jenkins-hbase4:43901] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 16:54:25,690 INFO [RS:2;jenkins-hbase4:43901] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,691 INFO [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 16:54:25,693 INFO [RS:2;jenkins-hbase4:43901] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,693 INFO [RS:1;jenkins-hbase4:46685] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 16:54:25,693 DEBUG [RS:2;jenkins-hbase4:43901] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,693 DEBUG [RS:2;jenkins-hbase4:43901] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,693 DEBUG [RS:2;jenkins-hbase4:43901] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,693 DEBUG [RS:2;jenkins-hbase4:43901] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,693 DEBUG [RS:2;jenkins-hbase4:43901] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,693 DEBUG [RS:2;jenkins-hbase4:43901] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 16:54:25,693 DEBUG [RS:2;jenkins-hbase4:43901] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,693 DEBUG [RS:2;jenkins-hbase4:43901] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,693 DEBUG [RS:2;jenkins-hbase4:43901] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,693 DEBUG [RS:2;jenkins-hbase4:43901] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,695 INFO [RS:1;jenkins-hbase4:46685] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 16:54:25,695 INFO [RS:1;jenkins-hbase4:46685] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,696 INFO [RS:3;jenkins-hbase4:45821] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 16:54:25,697 INFO [RS:1;jenkins-hbase4:46685] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 16:54:25,697 INFO [RS:0;jenkins-hbase4:45493] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 16:54:25,697 INFO [RS:2;jenkins-hbase4:43901] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,697 INFO [RS:2;jenkins-hbase4:43901] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,697 INFO [RS:2;jenkins-hbase4:43901] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,697 INFO [RS:3;jenkins-hbase4:45821] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 16:54:25,698 INFO [RS:3;jenkins-hbase4:45821] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,698 INFO [RS:0;jenkins-hbase4:45493] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 16:54:25,699 INFO [RS:0;jenkins-hbase4:45493] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,700 INFO [RS:1;jenkins-hbase4:46685] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,700 INFO [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 16:54:25,702 INFO [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 16:54:25,703 DEBUG [RS:1;jenkins-hbase4:46685] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,705 DEBUG [RS:1;jenkins-hbase4:46685] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,705 DEBUG [RS:1;jenkins-hbase4:46685] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,705 DEBUG [RS:1;jenkins-hbase4:46685] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,705 DEBUG [RS:1;jenkins-hbase4:46685] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,705 DEBUG [RS:1;jenkins-hbase4:46685] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 16:54:25,705 DEBUG [RS:1;jenkins-hbase4:46685] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,705 DEBUG [RS:1;jenkins-hbase4:46685] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,705 DEBUG [RS:1;jenkins-hbase4:46685] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,706 DEBUG [RS:1;jenkins-hbase4:46685] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,706 INFO [RS:0;jenkins-hbase4:45493] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,706 INFO [RS:3;jenkins-hbase4:45821] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,710 DEBUG [RS:0;jenkins-hbase4:45493] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,712 INFO [RS:1;jenkins-hbase4:46685] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,712 DEBUG [RS:0;jenkins-hbase4:45493] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,712 INFO [RS:1;jenkins-hbase4:46685] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,713 DEBUG [RS:0;jenkins-hbase4:45493] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,711 DEBUG [RS:3;jenkins-hbase4:45821] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,713 DEBUG [RS:0;jenkins-hbase4:45493] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,713 DEBUG [RS:3;jenkins-hbase4:45821] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,713 INFO [RS:1;jenkins-hbase4:46685] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,713 DEBUG [RS:0;jenkins-hbase4:45493] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,713 DEBUG [RS:3;jenkins-hbase4:45821] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,713 DEBUG [RS:0;jenkins-hbase4:45493] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 16:54:25,713 DEBUG [RS:3;jenkins-hbase4:45821] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,713 DEBUG [RS:0;jenkins-hbase4:45493] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,713 DEBUG [RS:3;jenkins-hbase4:45821] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,713 DEBUG [RS:0;jenkins-hbase4:45493] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,713 DEBUG [RS:3;jenkins-hbase4:45821] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 16:54:25,714 DEBUG [RS:0;jenkins-hbase4:45493] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,714 DEBUG [RS:3;jenkins-hbase4:45821] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,714 DEBUG [RS:0;jenkins-hbase4:45493] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,714 DEBUG [RS:3;jenkins-hbase4:45821] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,714 DEBUG [RS:3;jenkins-hbase4:45821] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,714 DEBUG [RS:3;jenkins-hbase4:45821] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:25,721 INFO [RS:0;jenkins-hbase4:45493] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,721 INFO [RS:0;jenkins-hbase4:45493] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,721 INFO [RS:0;jenkins-hbase4:45493] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,721 INFO [RS:3;jenkins-hbase4:45821] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,721 INFO [RS:3;jenkins-hbase4:45821] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,721 INFO [RS:3;jenkins-hbase4:45821] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,723 INFO [RS:2;jenkins-hbase4:43901] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 16:54:25,723 INFO [RS:2;jenkins-hbase4:43901] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43901,1689872065207-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,730 INFO [RS:1;jenkins-hbase4:46685] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 16:54:25,730 INFO [RS:1;jenkins-hbase4:46685] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46685,1689872065178-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,735 INFO [RS:0;jenkins-hbase4:45493] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 16:54:25,735 INFO [RS:0;jenkins-hbase4:45493] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45493,1689872065148-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,737 INFO [RS:3;jenkins-hbase4:45821] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 16:54:25,737 INFO [RS:3;jenkins-hbase4:45821] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45821,1689872065238-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:25,741 INFO [RS:2;jenkins-hbase4:43901] regionserver.Replication(203): jenkins-hbase4.apache.org,43901,1689872065207 started 2023-07-20 16:54:25,741 INFO [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,43901,1689872065207, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43901, sessionid=0x10183a7c131000a 2023-07-20 16:54:25,741 DEBUG [RS:2;jenkins-hbase4:43901] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 16:54:25,741 DEBUG [RS:2;jenkins-hbase4:43901] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:25,741 DEBUG [RS:2;jenkins-hbase4:43901] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43901,1689872065207' 2023-07-20 16:54:25,741 DEBUG [RS:2;jenkins-hbase4:43901] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-20 16:54:25,741 DEBUG [RS:2;jenkins-hbase4:43901] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-20 16:54:25,743 DEBUG [RS:2;jenkins-hbase4:43901] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 16:54:25,743 DEBUG [RS:2;jenkins-hbase4:43901] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 16:54:25,743 DEBUG [RS:2;jenkins-hbase4:43901] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:25,743 DEBUG [RS:2;jenkins-hbase4:43901] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43901,1689872065207' 2023-07-20 16:54:25,743 DEBUG [RS:2;jenkins-hbase4:43901] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-20 16:54:25,744 DEBUG [RS:2;jenkins-hbase4:43901] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-20 16:54:25,744 DEBUG [RS:2;jenkins-hbase4:43901] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 16:54:25,744 INFO [RS:2;jenkins-hbase4:43901] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 16:54:25,744 INFO [RS:2;jenkins-hbase4:43901] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 16:54:25,746 INFO [RS:1;jenkins-hbase4:46685] regionserver.Replication(203): jenkins-hbase4.apache.org,46685,1689872065178 started 2023-07-20 16:54:25,746 INFO [RS:1;jenkins-hbase4:46685] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46685,1689872065178, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46685, sessionid=0x10183a7c1310009 2023-07-20 16:54:25,746 DEBUG [RS:1;jenkins-hbase4:46685] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 16:54:25,746 DEBUG [RS:1;jenkins-hbase4:46685] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:25,746 DEBUG [RS:1;jenkins-hbase4:46685] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46685,1689872065178' 2023-07-20 16:54:25,746 DEBUG [RS:1;jenkins-hbase4:46685] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-20 16:54:25,747 DEBUG [RS:1;jenkins-hbase4:46685] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-20 16:54:25,747 DEBUG [RS:1;jenkins-hbase4:46685] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 16:54:25,747 DEBUG [RS:1;jenkins-hbase4:46685] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 16:54:25,747 DEBUG [RS:1;jenkins-hbase4:46685] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:25,747 DEBUG [RS:1;jenkins-hbase4:46685] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46685,1689872065178' 2023-07-20 16:54:25,747 DEBUG [RS:1;jenkins-hbase4:46685] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-20 16:54:25,748 DEBUG [RS:1;jenkins-hbase4:46685] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-20 16:54:25,748 DEBUG [RS:1;jenkins-hbase4:46685] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 16:54:25,748 INFO [RS:1;jenkins-hbase4:46685] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 16:54:25,748 INFO [RS:1;jenkins-hbase4:46685] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 16:54:25,750 INFO [RS:0;jenkins-hbase4:45493] regionserver.Replication(203): jenkins-hbase4.apache.org,45493,1689872065148 started 2023-07-20 16:54:25,750 INFO [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,45493,1689872065148, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45493, sessionid=0x10183a7c1310008 2023-07-20 16:54:25,750 DEBUG [RS:0;jenkins-hbase4:45493] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 16:54:25,750 DEBUG [RS:0;jenkins-hbase4:45493] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:25,750 DEBUG [RS:0;jenkins-hbase4:45493] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45493,1689872065148' 2023-07-20 16:54:25,750 DEBUG [RS:0;jenkins-hbase4:45493] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-20 16:54:25,751 DEBUG [RS:0;jenkins-hbase4:45493] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-20 16:54:25,751 DEBUG [RS:0;jenkins-hbase4:45493] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 16:54:25,751 DEBUG [RS:0;jenkins-hbase4:45493] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 16:54:25,751 DEBUG [RS:0;jenkins-hbase4:45493] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:25,751 DEBUG [RS:0;jenkins-hbase4:45493] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45493,1689872065148' 2023-07-20 16:54:25,751 DEBUG [RS:0;jenkins-hbase4:45493] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-20 16:54:25,751 INFO [RS:3;jenkins-hbase4:45821] regionserver.Replication(203): jenkins-hbase4.apache.org,45821,1689872065238 started 2023-07-20 16:54:25,751 INFO [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,45821,1689872065238, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45821, sessionid=0x10183a7c131000b 2023-07-20 16:54:25,752 DEBUG [RS:3;jenkins-hbase4:45821] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 16:54:25,752 DEBUG [RS:3;jenkins-hbase4:45821] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:25,752 DEBUG [RS:3;jenkins-hbase4:45821] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45821,1689872065238' 2023-07-20 16:54:25,752 DEBUG [RS:0;jenkins-hbase4:45493] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-20 16:54:25,752 DEBUG [RS:3;jenkins-hbase4:45821] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-20 16:54:25,752 DEBUG [RS:0;jenkins-hbase4:45493] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 16:54:25,752 DEBUG [RS:3;jenkins-hbase4:45821] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-20 16:54:25,752 INFO [RS:0;jenkins-hbase4:45493] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 16:54:25,752 INFO [RS:0;jenkins-hbase4:45493] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 16:54:25,753 DEBUG [RS:3;jenkins-hbase4:45821] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 16:54:25,753 DEBUG [RS:3;jenkins-hbase4:45821] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 16:54:25,753 DEBUG [RS:3;jenkins-hbase4:45821] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:25,753 DEBUG [RS:3;jenkins-hbase4:45821] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45821,1689872065238' 2023-07-20 16:54:25,753 DEBUG [RS:3;jenkins-hbase4:45821] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-20 16:54:25,753 DEBUG [RS:3;jenkins-hbase4:45821] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-20 16:54:25,754 DEBUG [RS:3;jenkins-hbase4:45821] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 16:54:25,754 INFO [RS:3;jenkins-hbase4:45821] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 16:54:25,754 INFO [RS:3;jenkins-hbase4:45821] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 16:54:25,786 DEBUG [jenkins-hbase4:45867] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=4, allServersCount=4 2023-07-20 16:54:25,787 DEBUG [jenkins-hbase4:45867] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 16:54:25,789 DEBUG [jenkins-hbase4:45867] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 16:54:25,789 DEBUG [jenkins-hbase4:45867] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 16:54:25,789 DEBUG [jenkins-hbase4:45867] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-20 16:54:25,789 DEBUG [jenkins-hbase4:45867] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-20 16:54:25,789 DEBUG [jenkins-hbase4:45867] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 16:54:25,791 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45493,1689872065148, state=OPENING 2023-07-20 16:54:25,792 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-20 16:54:25,794 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:54:25,794 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-20 16:54:25,794 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45493,1689872065148}] 2023-07-20 16:54:25,844 INFO [RS:2;jenkins-hbase4:43901] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:25,847 INFO [RS:2;jenkins-hbase4:43901] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43901%2C1689872065207.jenkins-hbase4.apache.org%2C43901%2C1689872065207.regiongroup-0, suffix=, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,43901,1689872065207, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs, maxLogs=10 2023-07-20 16:54:25,848 INFO [RS:1;jenkins-hbase4:46685] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:25,851 INFO [RS:1;jenkins-hbase4:46685] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46685%2C1689872065178.jenkins-hbase4.apache.org%2C46685%2C1689872065178.regiongroup-0, suffix=, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46685,1689872065178, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs, maxLogs=10 2023-07-20 16:54:25,853 INFO [RS:0;jenkins-hbase4:45493] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:25,856 INFO [RS:3;jenkins-hbase4:45821] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:25,865 INFO [RS:0;jenkins-hbase4:45493] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45493%2C1689872065148.jenkins-hbase4.apache.org%2C45493%2C1689872065148.regiongroup-0, suffix=, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs, maxLogs=10 2023-07-20 16:54:25,866 INFO [RS:3;jenkins-hbase4:45821] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45821%2C1689872065238.jenkins-hbase4.apache.org%2C45821%2C1689872065238.regiongroup-0, suffix=, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45821,1689872065238, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs, maxLogs=10 2023-07-20 16:54:25,917 INFO [RS:2;jenkins-hbase4:43901] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,43901,1689872065207/jenkins-hbase4.apache.org%2C43901%2C1689872065207.jenkins-hbase4.apache.org%2C43901%2C1689872065207.regiongroup-0.1689872065849 2023-07-20 16:54:25,918 DEBUG [RS:2;jenkins-hbase4:43901] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36101,DS-69ae4973-7589-41c7-bb37-0af0ca263f5e,DISK], DatanodeInfoWithStorage[127.0.0.1:35885,DS-21386278-ed61-414b-9d04-87c474a58f9b,DISK], DatanodeInfoWithStorage[127.0.0.1:35847,DS-e3134465-ba87-4999-ba03-bbcb207f3c3b,DISK]] 2023-07-20 16:54:25,918 INFO [RS:1;jenkins-hbase4:46685] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46685,1689872065178/jenkins-hbase4.apache.org%2C46685%2C1689872065178.jenkins-hbase4.apache.org%2C46685%2C1689872065178.regiongroup-0.1689872065857 2023-07-20 16:54:25,919 DEBUG [RS:1;jenkins-hbase4:46685] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36101,DS-69ae4973-7589-41c7-bb37-0af0ca263f5e,DISK], DatanodeInfoWithStorage[127.0.0.1:35885,DS-21386278-ed61-414b-9d04-87c474a58f9b,DISK], DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK]] 2023-07-20 16:54:25,952 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:25,952 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 16:54:25,960 INFO [RS:0;jenkins-hbase4:45493] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148/jenkins-hbase4.apache.org%2C45493%2C1689872065148.jenkins-hbase4.apache.org%2C45493%2C1689872065148.regiongroup-0.1689872065870 2023-07-20 16:54:25,960 DEBUG [RS:0;jenkins-hbase4:45493] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36101,DS-69ae4973-7589-41c7-bb37-0af0ca263f5e,DISK], DatanodeInfoWithStorage[127.0.0.1:35847,DS-e3134465-ba87-4999-ba03-bbcb207f3c3b,DISK], DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK]] 2023-07-20 16:54:25,961 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53884, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 16:54:25,967 INFO [RS:3;jenkins-hbase4:45821] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45821,1689872065238/jenkins-hbase4.apache.org%2C45821%2C1689872065238.jenkins-hbase4.apache.org%2C45821%2C1689872065238.regiongroup-0.1689872065887 2023-07-20 16:54:25,968 DEBUG [RS:3;jenkins-hbase4:45821] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35847,DS-e3134465-ba87-4999-ba03-bbcb207f3c3b,DISK], DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK], DatanodeInfoWithStorage[127.0.0.1:36101,DS-69ae4973-7589-41c7-bb37-0af0ca263f5e,DISK]] 2023-07-20 16:54:25,969 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-20 16:54:25,969 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:54:25,969 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:54:25,969 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:25,971 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45493%2C1689872065148.meta, suffix=.meta, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs, maxLogs=10 2023-07-20 16:54:25,992 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148/jenkins-hbase4.apache.org%2C45493%2C1689872065148.meta.1689872065974.meta 2023-07-20 16:54:25,992 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK], DatanodeInfoWithStorage[127.0.0.1:35847,DS-e3134465-ba87-4999-ba03-bbcb207f3c3b,DISK], DatanodeInfoWithStorage[127.0.0.1:36101,DS-69ae4973-7589-41c7-bb37-0af0ca263f5e,DISK]] 2023-07-20 16:54:25,992 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:25,993 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-20 16:54:25,993 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-20 16:54:25,994 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-20 16:54:25,994 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-20 16:54:25,994 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:25,994 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-20 16:54:25,994 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-20 16:54:25,996 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-20 16:54:25,997 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info 2023-07-20 16:54:25,997 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info 2023-07-20 16:54:25,998 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-20 16:54:25,999 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:25,999 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-20 16:54:26,000 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/rep_barrier 2023-07-20 16:54:26,000 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/rep_barrier 2023-07-20 16:54:26,000 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-20 16:54:26,001 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:26,001 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-20 16:54:26,003 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/table 2023-07-20 16:54:26,003 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/table 2023-07-20 16:54:26,003 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-20 16:54:26,004 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:26,006 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740 2023-07-20 16:54:26,008 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740 2023-07-20 16:54:26,012 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-20 16:54:26,014 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-20 16:54:26,014 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70492160, jitterRate=0.0504150390625}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-20 16:54:26,015 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-20 16:54:26,017 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689872065952 2023-07-20 16:54:26,024 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-20 16:54:26,025 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-20 16:54:26,026 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45493,1689872065148, state=OPEN 2023-07-20 16:54:26,029 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-20 16:54:26,029 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-20 16:54:26,033 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-20 16:54:26,033 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45493,1689872065148 in 235 msec 2023-07-20 16:54:26,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-20 16:54:26,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 404 msec 2023-07-20 16:54:26,041 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 551 msec 2023-07-20 16:54:26,042 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689872066041, completionTime=-1 2023-07-20 16:54:26,042 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=4; waited=0ms, expected min=4 server(s), max=4 server(s), master is running 2023-07-20 16:54:26,042 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-20 16:54:26,046 DEBUG [hconnection-0xd610e78-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:26,050 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53896, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:54:26,052 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=4 2023-07-20 16:54:26,052 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689872126052 2023-07-20 16:54:26,052 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689872186052 2023-07-20 16:54:26,052 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 10 msec 2023-07-20 16:54:26,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45867,1689872065106-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:26,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45867,1689872065106-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:26,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45867,1689872065106-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:26,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:45867, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:26,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:26,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-20 16:54:26,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-20 16:54:26,064 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-20 16:54:26,064 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-20 16:54:26,067 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-20 16:54:26,069 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-20 16:54:26,071 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:26,072 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20 empty. 2023-07-20 16:54:26,073 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:26,073 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-20 16:54:26,095 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-20 16:54:26,096 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 7ebeef24b783c4189ed905a5659b7e20, NAME => 'hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp 2023-07-20 16:54:26,112 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:26,112 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 7ebeef24b783c4189ed905a5659b7e20, disabling compactions & flushes 2023-07-20 16:54:26,112 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:26,112 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:26,112 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. after waiting 0 ms 2023-07-20 16:54:26,112 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:26,112 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:26,112 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 7ebeef24b783c4189ed905a5659b7e20: 2023-07-20 16:54:26,117 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-20 16:54:26,118 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689872066118"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689872066118"}]},"ts":"1689872066118"} 2023-07-20 16:54:26,123 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-20 16:54:26,124 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-20 16:54:26,125 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689872066124"}]},"ts":"1689872066124"} 2023-07-20 16:54:26,127 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-20 16:54:26,133 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 16:54:26,134 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 16:54:26,134 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 16:54:26,134 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-20 16:54:26,134 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-20 16:54:26,134 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 16:54:26,135 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=7ebeef24b783c4189ed905a5659b7e20, ASSIGN}] 2023-07-20 16:54:26,139 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=7ebeef24b783c4189ed905a5659b7e20, ASSIGN 2023-07-20 16:54:26,140 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=7ebeef24b783c4189ed905a5659b7e20, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45821,1689872065238; forceNewPlan=false, retain=false 2023-07-20 16:54:26,291 INFO [jenkins-hbase4:45867] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-20 16:54:26,293 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=7ebeef24b783c4189ed905a5659b7e20, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:26,293 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689872066293"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872066293"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872066293"}]},"ts":"1689872066293"} 2023-07-20 16:54:26,297 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 7ebeef24b783c4189ed905a5659b7e20, server=jenkins-hbase4.apache.org,45821,1689872065238}] 2023-07-20 16:54:26,453 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:26,453 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 16:54:26,456 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56184, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 16:54:26,463 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:26,463 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:26,465 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45821%2C1689872065238.jenkins-hbase4.apache.org%2C45821%2C1689872065238.regiongroup-1, suffix=, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45821,1689872065238, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs, maxLogs=10 2023-07-20 16:54:26,483 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45821,1689872065238/jenkins-hbase4.apache.org%2C45821%2C1689872065238.jenkins-hbase4.apache.org%2C45821%2C1689872065238.regiongroup-1.1689872066465 2023-07-20 16:54:26,483 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK], DatanodeInfoWithStorage[127.0.0.1:36101,DS-69ae4973-7589-41c7-bb37-0af0ca263f5e,DISK], DatanodeInfoWithStorage[127.0.0.1:35885,DS-21386278-ed61-414b-9d04-87c474a58f9b,DISK]] 2023-07-20 16:54:26,483 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7ebeef24b783c4189ed905a5659b7e20, NAME => 'hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:26,484 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:26,484 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:26,484 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:26,484 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:26,486 INFO [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:26,489 DEBUG [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/info 2023-07-20 16:54:26,489 DEBUG [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/info 2023-07-20 16:54:26,490 INFO [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7ebeef24b783c4189ed905a5659b7e20 columnFamilyName info 2023-07-20 16:54:26,491 INFO [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] regionserver.HStore(310): Store=7ebeef24b783c4189ed905a5659b7e20/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:26,492 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:26,494 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:26,497 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:26,500 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 16:54:26,501 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7ebeef24b783c4189ed905a5659b7e20; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61707236, jitterRate=-0.08049052953720093}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 16:54:26,501 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7ebeef24b783c4189ed905a5659b7e20: 2023-07-20 16:54:26,504 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20., pid=6, masterSystemTime=1689872066453 2023-07-20 16:54:26,509 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:26,510 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:26,511 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=7ebeef24b783c4189ed905a5659b7e20, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:26,512 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689872066511"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872066511"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872066511"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872066511"}]},"ts":"1689872066511"} 2023-07-20 16:54:26,519 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-20 16:54:26,519 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 7ebeef24b783c4189ed905a5659b7e20, server=jenkins-hbase4.apache.org,45821,1689872065238 in 218 msec 2023-07-20 16:54:26,523 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-20 16:54:26,524 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=7ebeef24b783c4189ed905a5659b7e20, ASSIGN in 384 msec 2023-07-20 16:54:26,526 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-20 16:54:26,526 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689872066526"}]},"ts":"1689872066526"} 2023-07-20 16:54:26,529 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-20 16:54:26,532 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-20 16:54:26,535 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 470 msec 2023-07-20 16:54:26,565 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/namespace 2023-07-20 16:54:26,572 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/namespace 2023-07-20 16:54:26,572 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:54:26,576 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:26,581 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56194, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:54:26,584 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-20 16:54:26,599 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-20 16:54:26,604 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 19 msec 2023-07-20 16:54:26,607 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-20 16:54:26,619 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/namespace 2023-07-20 16:54:26,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-07-20 16:54:26,632 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-20 16:54:26,635 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-20 16:54:26,635 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.359sec 2023-07-20 16:54:26,635 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-20 16:54:26,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-20 16:54:26,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-20 16:54:26,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45867,1689872065106-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-20 16:54:26,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45867,1689872065106-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-20 16:54:26,641 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-20 16:54:26,666 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(139): Connect 0x0e1a2d76 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:26,674 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d9a3942, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:26,676 DEBUG [hconnection-0x59ddbaa7-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:26,679 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53904, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:54:26,682 INFO [Listener at localhost/33489] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,45867,1689872065106 2023-07-20 16:54:26,695 DEBUG [Listener at localhost/33489] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-20 16:54:26,698 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56304, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-20 16:54:26,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 16:54:26,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-20 16:54:26,719 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-20 16:54:26,721 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-20 16:54:26,724 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-20 16:54:26,724 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:54:26,725 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a empty. 2023-07-20 16:54:26,726 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:54:26,726 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-20 16:54:26,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 16:54:26,750 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-20 16:54:26,751 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => ea29850f58e32db8bff448bce6d2160a, NAME => 't1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp 2023-07-20 16:54:26,765 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:26,765 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing ea29850f58e32db8bff448bce6d2160a, disabling compactions & flushes 2023-07-20 16:54:26,765 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:54:26,765 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:54:26,765 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. after waiting 0 ms 2023-07-20 16:54:26,765 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:54:26,765 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:54:26,765 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for ea29850f58e32db8bff448bce6d2160a: 2023-07-20 16:54:26,769 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-20 16:54:26,771 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872066771"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689872066771"}]},"ts":"1689872066771"} 2023-07-20 16:54:26,773 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-20 16:54:26,775 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-20 16:54:26,775 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689872066775"}]},"ts":"1689872066775"} 2023-07-20 16:54:26,777 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-20 16:54:26,783 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 16:54:26,784 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 16:54:26,784 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 16:54:26,784 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 16:54:26,784 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=ea29850f58e32db8bff448bce6d2160a, ASSIGN}] 2023-07-20 16:54:26,787 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=ea29850f58e32db8bff448bce6d2160a, ASSIGN 2023-07-20 16:54:26,788 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=ea29850f58e32db8bff448bce6d2160a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33103,1689872061305; forceNewPlan=false, retain=false 2023-07-20 16:54:26,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 16:54:26,938 INFO [jenkins-hbase4:39639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-20 16:54:26,940 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=ea29850f58e32db8bff448bce6d2160a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:26,940 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872066940"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872066940"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872066940"}]},"ts":"1689872066940"} 2023-07-20 16:54:26,943 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure ea29850f58e32db8bff448bce6d2160a, server=jenkins-hbase4.apache.org,33103,1689872061305}] 2023-07-20 16:54:27,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 16:54:27,098 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:27,098 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 16:54:27,108 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52800, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 16:54:27,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:54:27,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:27,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1, suffix=, logDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305, archiveDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs, maxLogs=10 2023-07-20 16:54:27,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1.1689872067118 2023-07-20 16:54:27,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45027,DS-87ab81e2-6016-492e-a423-b308cc471afd,DISK], DatanodeInfoWithStorage[127.0.0.1:40699,DS-82a10023-4ecc-44e1-b86d-cd4749f69898,DISK]] 2023-07-20 16:54:27,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ea29850f58e32db8bff448bce6d2160a, NAME => 't1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:27,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:54:27,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:27,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:54:27,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:54:27,135 INFO [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:54:27,138 DEBUG [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/cf1 2023-07-20 16:54:27,138 DEBUG [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/cf1 2023-07-20 16:54:27,139 INFO [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea29850f58e32db8bff448bce6d2160a columnFamilyName cf1 2023-07-20 16:54:27,140 INFO [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] regionserver.HStore(310): Store=ea29850f58e32db8bff448bce6d2160a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:27,140 INFO [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:54:27,142 DEBUG [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/norep 2023-07-20 16:54:27,142 DEBUG [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/norep 2023-07-20 16:54:27,142 INFO [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea29850f58e32db8bff448bce6d2160a columnFamilyName norep 2023-07-20 16:54:27,143 INFO [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] regionserver.HStore(310): Store=ea29850f58e32db8bff448bce6d2160a/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:27,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:54:27,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:54:27,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 16:54:27,151 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:54:27,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 16:54:27,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ea29850f58e32db8bff448bce6d2160a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68916328, jitterRate=0.02693331241607666}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 16:54:27,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ea29850f58e32db8bff448bce6d2160a: 2023-07-20 16:54:27,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a., pid=11, masterSystemTime=1689872067098 2023-07-20 16:54:27,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:54:27,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:54:27,164 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=ea29850f58e32db8bff448bce6d2160a, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:27,164 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872067164"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872067164"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872067164"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872067164"}]},"ts":"1689872067164"} 2023-07-20 16:54:27,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-20 16:54:27,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure ea29850f58e32db8bff448bce6d2160a, server=jenkins-hbase4.apache.org,33103,1689872061305 in 224 msec 2023-07-20 16:54:27,174 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-20 16:54:27,175 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=ea29850f58e32db8bff448bce6d2160a, ASSIGN in 387 msec 2023-07-20 16:54:27,176 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-20 16:54:27,176 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689872067176"}]},"ts":"1689872067176"} 2023-07-20 16:54:27,178 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-20 16:54:27,181 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-20 16:54:27,184 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 467 msec 2023-07-20 16:54:27,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 16:54:27,353 INFO [Listener at localhost/33489] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-20 16:54:27,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 16:54:27,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-20 16:54:27,359 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-20 16:54:27,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-20 16:54:27,361 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-20 16:54:27,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 16:54:27,364 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:54:27,364 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705 empty. 2023-07-20 16:54:27,365 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:54:27,365 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-20 16:54:27,383 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-20 16:54:27,384 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2627e10c9cf6db0694d7b55799cf6705, NAME => 't2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.tmp 2023-07-20 16:54:27,399 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:27,399 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing 2627e10c9cf6db0694d7b55799cf6705, disabling compactions & flushes 2023-07-20 16:54:27,399 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:54:27,399 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:54:27,399 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. after waiting 0 ms 2023-07-20 16:54:27,399 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:54:27,399 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:54:27,399 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 2627e10c9cf6db0694d7b55799cf6705: 2023-07-20 16:54:27,403 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-20 16:54:27,405 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872067405"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689872067405"}]},"ts":"1689872067405"} 2023-07-20 16:54:27,408 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-20 16:54:27,409 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-20 16:54:27,409 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689872067409"}]},"ts":"1689872067409"} 2023-07-20 16:54:27,412 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-20 16:54:27,416 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 16:54:27,417 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 16:54:27,417 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 16:54:27,418 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 16:54:27,418 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=2627e10c9cf6db0694d7b55799cf6705, ASSIGN}] 2023-07-20 16:54:27,420 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=2627e10c9cf6db0694d7b55799cf6705, ASSIGN 2023-07-20 16:54:27,422 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=2627e10c9cf6db0694d7b55799cf6705, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33103,1689872061305; forceNewPlan=false, retain=false 2023-07-20 16:54:27,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 16:54:27,572 INFO [jenkins-hbase4:39639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-20 16:54:27,574 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=2627e10c9cf6db0694d7b55799cf6705, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:27,574 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872067574"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872067574"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872067574"}]},"ts":"1689872067574"} 2023-07-20 16:54:27,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 2627e10c9cf6db0694d7b55799cf6705, server=jenkins-hbase4.apache.org,33103,1689872061305}] 2023-07-20 16:54:27,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 16:54:27,693 INFO [jenkins-hbase4:34183Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:54:27,693 INFO [jenkins-hbase4:33103Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:54:27,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:54:27,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2627e10c9cf6db0694d7b55799cf6705, NAME => 't2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:27,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:54:27,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:27,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:54:27,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:54:27,737 INFO [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:54:27,739 DEBUG [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/cf1 2023-07-20 16:54:27,739 DEBUG [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/cf1 2023-07-20 16:54:27,740 INFO [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2627e10c9cf6db0694d7b55799cf6705 columnFamilyName cf1 2023-07-20 16:54:27,740 INFO [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] regionserver.HStore(310): Store=2627e10c9cf6db0694d7b55799cf6705/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:27,740 INFO [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:54:27,742 DEBUG [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/norep 2023-07-20 16:54:27,742 DEBUG [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/norep 2023-07-20 16:54:27,742 INFO [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2627e10c9cf6db0694d7b55799cf6705 columnFamilyName norep 2023-07-20 16:54:27,743 INFO [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] regionserver.HStore(310): Store=2627e10c9cf6db0694d7b55799cf6705/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:27,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:54:27,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:54:27,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 16:54:27,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:54:27,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 16:54:27,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2627e10c9cf6db0694d7b55799cf6705; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71342067, jitterRate=0.06307964026927948}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 16:54:27,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2627e10c9cf6db0694d7b55799cf6705: 2023-07-20 16:54:27,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705., pid=14, masterSystemTime=1689872067730 2023-07-20 16:54:27,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:54:27,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:54:27,762 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=2627e10c9cf6db0694d7b55799cf6705, regionState=OPEN, repBarrier=2, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:27,762 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872067762"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872067762"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872067762"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872067762"}]},"ts":"1689872067762"} 2023-07-20 16:54:27,768 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-20 16:54:27,768 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 2627e10c9cf6db0694d7b55799cf6705, server=jenkins-hbase4.apache.org,33103,1689872061305 in 188 msec 2023-07-20 16:54:27,775 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-20 16:54:27,776 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=2627e10c9cf6db0694d7b55799cf6705, ASSIGN in 350 msec 2023-07-20 16:54:27,777 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-20 16:54:27,777 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689872067777"}]},"ts":"1689872067777"} 2023-07-20 16:54:27,780 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-20 16:54:27,784 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-20 16:54:27,787 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 429 msec 2023-07-20 16:54:27,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 16:54:27,967 INFO [Listener at localhost/33489] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-20 16:54:27,971 DEBUG [Listener at localhost/33489] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-20 16:54:27,975 INFO [RS-EventLoopGroup-6-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47376, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-20 16:54:27,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 16:54:27,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t1_syncup 2023-07-20 16:54:27,983 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-20 16:54:27,984 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t1_syncup" procId is: 9 2023-07-20 16:54:27,986 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-20 16:54:27,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 16:54:27,988 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:27,989 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277 empty. 2023-07-20 16:54:27,989 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:27,989 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived t1_syncup regions 2023-07-20 16:54:28,030 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp/data/default/t1_syncup/.tabledesc/.tableinfo.0000000001 2023-07-20 16:54:28,031 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 058a0bfd9d8bf39d8c91089a151cb277, NAME => 't1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t1_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp 2023-07-20 16:54:28,046 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:28,046 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1604): Closing 058a0bfd9d8bf39d8c91089a151cb277, disabling compactions & flushes 2023-07-20 16:54:28,046 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:28,046 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:28,047 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. after waiting 0 ms 2023-07-20 16:54:28,047 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:28,047 INFO [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1838): Closed t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:28,047 DEBUG [RegionOpenAndInit-t1_syncup-pool-0] regionserver.HRegion(1558): Region close journal for 058a0bfd9d8bf39d8c91089a151cb277: 2023-07-20 16:54:28,051 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-20 16:54:28,053 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872068053"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689872068053"}]},"ts":"1689872068053"} 2023-07-20 16:54:28,056 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-20 16:54:28,057 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-20 16:54:28,057 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689872068057"}]},"ts":"1689872068057"} 2023-07-20 16:54:28,059 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLING in hbase:meta 2023-07-20 16:54:28,064 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 16:54:28,067 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 16:54:28,067 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 16:54:28,067 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-20 16:54:28,067 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-20 16:54:28,067 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 16:54:28,067 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=058a0bfd9d8bf39d8c91089a151cb277, ASSIGN}] 2023-07-20 16:54:28,071 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=058a0bfd9d8bf39d8c91089a151cb277, ASSIGN 2023-07-20 16:54:28,072 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=058a0bfd9d8bf39d8c91089a151cb277, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45493,1689872065148; forceNewPlan=false, retain=false 2023-07-20 16:54:28,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 16:54:28,223 INFO [jenkins-hbase4:45867] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-20 16:54:28,224 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=058a0bfd9d8bf39d8c91089a151cb277, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:28,225 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872068224"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872068224"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872068224"}]},"ts":"1689872068224"} 2023-07-20 16:54:28,227 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 058a0bfd9d8bf39d8c91089a151cb277, server=jenkins-hbase4.apache.org,45493,1689872065148}] 2023-07-20 16:54:28,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 16:54:28,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:28,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:28,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45493%2C1689872065148.jenkins-hbase4.apache.org%2C45493%2C1689872065148.regiongroup-1, suffix=, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs, maxLogs=10 2023-07-20 16:54:28,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148/jenkins-hbase4.apache.org%2C45493%2C1689872065148.jenkins-hbase4.apache.org%2C45493%2C1689872065148.regiongroup-1.1689872068393 2023-07-20 16:54:28,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35847,DS-e3134465-ba87-4999-ba03-bbcb207f3c3b,DISK], DatanodeInfoWithStorage[127.0.0.1:35885,DS-21386278-ed61-414b-9d04-87c474a58f9b,DISK], DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK]] 2023-07-20 16:54:28,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 058a0bfd9d8bf39d8c91089a151cb277, NAME => 't1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:28,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:28,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:28,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:28,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:28,414 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:28,420 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1 2023-07-20 16:54:28,421 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1 2023-07-20 16:54:28,421 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 058a0bfd9d8bf39d8c91089a151cb277 columnFamilyName cf1 2023-07-20 16:54:28,422 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(310): Store=058a0bfd9d8bf39d8c91089a151cb277/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:28,422 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:28,423 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/norep 2023-07-20 16:54:28,423 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/norep 2023-07-20 16:54:28,424 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 058a0bfd9d8bf39d8c91089a151cb277 columnFamilyName norep 2023-07-20 16:54:28,424 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(310): Store=058a0bfd9d8bf39d8c91089a151cb277/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:28,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:28,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:28,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 16:54:28,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:28,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 16:54:28,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 058a0bfd9d8bf39d8c91089a151cb277; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64589674, jitterRate=-0.03753885626792908}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 16:54:28,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 058a0bfd9d8bf39d8c91089a151cb277: 2023-07-20 16:54:28,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277., pid=11, masterSystemTime=1689872068381 2023-07-20 16:54:28,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:28,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:28,439 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=058a0bfd9d8bf39d8c91089a151cb277, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:28,440 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872068439"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872068439"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872068439"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872068439"}]},"ts":"1689872068439"} 2023-07-20 16:54:28,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-20 16:54:28,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 058a0bfd9d8bf39d8c91089a151cb277, server=jenkins-hbase4.apache.org,45493,1689872065148 in 215 msec 2023-07-20 16:54:28,449 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-20 16:54:28,449 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=058a0bfd9d8bf39d8c91089a151cb277, ASSIGN in 378 msec 2023-07-20 16:54:28,450 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-20 16:54:28,450 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t1_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689872068450"}]},"ts":"1689872068450"} 2023-07-20 16:54:28,452 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=t1_syncup, state=ENABLED in hbase:meta 2023-07-20 16:54:28,456 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t1_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-20 16:54:28,458 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup in 478 msec 2023-07-20 16:54:28,521 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 16:54:28,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-07-20 16:54:28,591 INFO [Listener at localhost/33489] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t1_syncup, procId: 9 completed 2023-07-20 16:54:28,593 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 't2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 16:54:28,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=t2_syncup 2023-07-20 16:54:28,597 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-20 16:54:28,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "t2_syncup" procId is: 12 2023-07-20 16:54:28,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 16:54:28,600 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-20 16:54:28,603 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:28,605 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e empty. 2023-07-20 16:54:28,606 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:28,606 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived t2_syncup regions 2023-07-20 16:54:28,680 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp/data/default/t2_syncup/.tabledesc/.tableinfo.0000000001 2023-07-20 16:54:28,688 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(7675): creating {ENCODED => c96c52d963776c1249f8d28dc1cee95e, NAME => 't2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='t2_syncup', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'norep', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.tmp 2023-07-20 16:54:28,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 16:54:28,773 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-20 16:54:28,774 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-20 16:54:28,775 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-20 16:54:28,775 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-20 16:54:28,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 16:54:29,137 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:29,137 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1604): Closing c96c52d963776c1249f8d28dc1cee95e, disabling compactions & flushes 2023-07-20 16:54:29,137 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:29,137 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:29,137 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. after waiting 0 ms 2023-07-20 16:54:29,137 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:29,137 INFO [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1838): Closed t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:29,137 DEBUG [RegionOpenAndInit-t2_syncup-pool-0] regionserver.HRegion(1558): Region close journal for c96c52d963776c1249f8d28dc1cee95e: 2023-07-20 16:54:29,142 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ADD_TO_META 2023-07-20 16:54:29,143 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872069143"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689872069143"}]},"ts":"1689872069143"} 2023-07-20 16:54:29,145 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-20 16:54:29,147 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-20 16:54:29,147 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689872069147"}]},"ts":"1689872069147"} 2023-07-20 16:54:29,149 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLING in hbase:meta 2023-07-20 16:54:29,155 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-20 16:54:29,157 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-20 16:54:29,157 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-20 16:54:29,157 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-20 16:54:29,157 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-20 16:54:29,157 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-20 16:54:29,158 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=c96c52d963776c1249f8d28dc1cee95e, ASSIGN}] 2023-07-20 16:54:29,160 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=c96c52d963776c1249f8d28dc1cee95e, ASSIGN 2023-07-20 16:54:29,162 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=c96c52d963776c1249f8d28dc1cee95e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43901,1689872065207; forceNewPlan=false, retain=false 2023-07-20 16:54:29,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 16:54:29,312 INFO [jenkins-hbase4:45867] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-20 16:54:29,314 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=c96c52d963776c1249f8d28dc1cee95e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:29,314 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872069314"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872069314"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872069314"}]},"ts":"1689872069314"} 2023-07-20 16:54:29,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure c96c52d963776c1249f8d28dc1cee95e, server=jenkins-hbase4.apache.org,43901,1689872065207}] 2023-07-20 16:54:29,471 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:29,472 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 16:54:29,476 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49784, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 16:54:29,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:29,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:29,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43901%2C1689872065207.jenkins-hbase4.apache.org%2C43901%2C1689872065207.regiongroup-1, suffix=, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,43901,1689872065207, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs, maxLogs=10 2023-07-20 16:54:29,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,43901,1689872065207/jenkins-hbase4.apache.org%2C43901%2C1689872065207.jenkins-hbase4.apache.org%2C43901%2C1689872065207.regiongroup-1.1689872069483 2023-07-20 16:54:29,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35847,DS-e3134465-ba87-4999-ba03-bbcb207f3c3b,DISK], DatanodeInfoWithStorage[127.0.0.1:36101,DS-69ae4973-7589-41c7-bb37-0af0ca263f5e,DISK], DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK]] 2023-07-20 16:54:29,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c96c52d963776c1249f8d28dc1cee95e, NAME => 't2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:29,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:29,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:29,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:29,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:29,509 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:29,514 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1 2023-07-20 16:54:29,514 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1 2023-07-20 16:54:29,514 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c96c52d963776c1249f8d28dc1cee95e columnFamilyName cf1 2023-07-20 16:54:29,515 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(310): Store=c96c52d963776c1249f8d28dc1cee95e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:29,515 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:29,517 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/norep 2023-07-20 16:54:29,517 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/norep 2023-07-20 16:54:29,517 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c96c52d963776c1249f8d28dc1cee95e columnFamilyName norep 2023-07-20 16:54:29,518 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(310): Store=c96c52d963776c1249f8d28dc1cee95e/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:29,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:29,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:29,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 16:54:29,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:29,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-20 16:54:29,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c96c52d963776c1249f8d28dc1cee95e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65438330, jitterRate=-0.0248928964138031}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 16:54:29,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c96c52d963776c1249f8d28dc1cee95e: 2023-07-20 16:54:29,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e., pid=14, masterSystemTime=1689872069471 2023-07-20 16:54:29,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:29,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:29,535 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=c96c52d963776c1249f8d28dc1cee95e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:29,536 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872069535"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872069535"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872069535"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872069535"}]},"ts":"1689872069535"} 2023-07-20 16:54:29,541 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-20 16:54:29,542 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure c96c52d963776c1249f8d28dc1cee95e, server=jenkins-hbase4.apache.org,43901,1689872065207 in 221 msec 2023-07-20 16:54:29,545 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-20 16:54:29,545 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=c96c52d963776c1249f8d28dc1cee95e, ASSIGN in 384 msec 2023-07-20 16:54:29,546 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-20 16:54:29,546 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"t2_syncup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689872069546"}]},"ts":"1689872069546"} 2023-07-20 16:54:29,548 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=t2_syncup, state=ENABLED in hbase:meta 2023-07-20 16:54:29,551 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=t2_syncup execute state=CREATE_TABLE_POST_OPERATION 2023-07-20 16:54:29,553 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup in 958 msec 2023-07-20 16:54:29,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45867] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-07-20 16:54:29,710 INFO [Listener at localhost/33489] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:t2_syncup, procId: 12 completed 2023-07-20 16:54:29,710 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(139): Connect 0x4a3c513a to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:29,717 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c5ed229, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:29,718 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(139): Connect 0x21b950d4 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:29,733 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@9246058, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:29,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.HMaster(3464): Client=jenkins//172.31.14.131 creating replication peer, id=1, config=clusterKey=127.0.0.1:56032:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false, state=ENABLED 2023-07-20 16:54:29,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:PRE_PEER_MODIFICATION; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-20 16:54:29,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-20 16:54:29,774 INFO [PEWorker-1] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-20 16:54:29,774 INFO [PEWorker-1] zookeeper.RecoverableZooKeeper(93): Process identifier=org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@26860842check-peer-cluster-id connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:29,779 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@26860842check-peer-cluster-id0x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:29,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@26860842check-peer-cluster-id-0x10183a7c1310014 connected 2023-07-20 16:54:29,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-20 16:54:29,884 DEBUG [PEWorker-1-EventThread] zookeeper.ZKWatcher(600): org.apache.hadoop.hbase.master.replication.ReplicationPeerManager@26860842check-peer-cluster-id-0x10183a7c1310014, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:29,895 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:54:29,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=17, ppid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-07-20 16:54:30,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-20 16:54:30,076 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34183] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=17 2023-07-20 16:54:30,076 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33103] regionserver.RSRpcServices(3879): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=16 2023-07-20 16:54:30,078 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-20 16:54:30,078 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RefreshPeerCallable(54): Received a peer change event, peerId=1, type=ADD_PEER 2023-07-20 16:54:30,097 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-20 16:54:30,097 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-20 16:54:30,124 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-20 16:54:30,124 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-20 16:54:30,158 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=17 2023-07-20 16:54:30,158 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0] regionserver.RemoteProcedureResultReporter(64): Successfully complete execution of pid=16 2023-07-20 16:54:30,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39639] master.HMaster(3641): Remote procedure done, pid=17 2023-07-20 16:54:30,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.HMaster(3641): Remote procedure done, pid=16 2023-07-20 16:54:30,161 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39639] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,34183,1689872061357 suceeded 2023-07-20 16:54:30,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] replication.RefreshPeerProcedure(116): Refresh peer 1 for ADD on jenkins-hbase4.apache.org,33103,1689872061305 suceeded 2023-07-20 16:54:30,167 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 257 msec 2023-07-20 16:54:30,169 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-07-20 16:54:30,169 INFO [PEWorker-3] replication.AddPeerProcedure(102): Successfully added ENABLED peer 1, config clusterKey=127.0.0.1:56032:/2,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false 2023-07-20 16:54:30,169 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 258 msec 2023-07-20 16:54:30,171 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure in 420 msec 2023-07-20 16:54:30,186 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x67310515 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:30,192 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x05efb24f to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:30,197 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@7c673c8f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:30,198 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@7a76ab6f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:30,200 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:30,201 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:30,206 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:30,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10183a7c1310017 connected 2023-07-20 16:54:30,207 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:30,208 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=258804e0-584b-4fe6-b46a-945d0467644e to cluster=de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:54:30,208 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=2) is replicating from cluster=258804e0-584b-4fe6-b46a-945d0467644e to cluster=de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:54:30,211 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10183a7c1310018 connected 2023-07-20 16:54:30,212 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0 2023-07-20 16:54:30,212 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1 2023-07-20 16:54:30,221 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 16:54:30,221 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 16:54:30,222 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0 2023-07-20 16:54:30,222 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1 2023-07-20 16:54:30,222 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0 2023-07-20 16:54:30,222 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 16:54:30,222 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1 2023-07-20 16:54:30,222 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 16:54:30,228 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0 2023-07-20 16:54:30,228 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1 2023-07-20 16:54:30,229 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-20 16:54:30,229 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-20 16:54:30,323 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:30,329 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-20 16:54:30,329 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-20 16:54:30,329 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 16:54:30,329 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 16:54:30,329 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0.1689872062821, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 16:54:30,329 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-20 16:54:30,329 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1.1689872067118, lastWalPosition=384, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 16:54:30,329 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:30,329 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-20 16:54:30,329 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 16:54:30,329 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:30,329 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357/jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1.1689872063702, lastWalPosition=600, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 16:54:30,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-07-20 16:54:30,368 DEBUG [Listener at localhost/33489] replication.TestReplicationSyncUpTool(91): putAndReplicateRows 2023-07-20 16:54:30,370 DEBUG [hconnection-0x7ee09fdc-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:30,374 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58078, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:54:30,379 DEBUG [Listener at localhost/33489] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:30,384 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43518, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:54:30,435 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:30,448 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:30,448 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-20 16:54:30,455 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 2989 2023-07-20 16:54:30,455 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 25 WAL entries eligible for replication 2023-07-20 16:54:30,456 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/ea29850f58e32db8bff448bce6d2160a/4=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/5=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/6=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/7=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/8=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/9=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/10=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/11=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/12=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/13=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/14=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/15=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/16=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/17=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/18=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/19=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/20=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/21=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/22=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/23=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/24=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/25=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/26=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/27=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/28=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1.1689872067118, lastWalPosition=2989, nbRowKeys=25, nbHFiles=0, heapSize=5200, lastSeqIds={}, endOfFile=false,usedBufferSize=5200] 2023-07-20 16:54:30,458 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 2989 2023-07-20 16:54:30,474 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 16:54:30,480 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48556, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-07-20 16:54:30,484 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46685] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 16:54:30,485 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46685] zookeeper.ReadOnlyZKClient(139): Connect 0x52bed061 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:30,503 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46685] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@5aef3a01, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:30,513 DEBUG [hconnection-0x338ac67-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:30,518 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55562, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-07-20 16:54:30,546 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:30,576 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:30,576 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 384 2023-07-20 16:54:30,598 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 8659 2023-07-20 16:54:30,598 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 54 WAL entries eligible for replication 2023-07-20 16:54:30,599 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 8659 2023-07-20 16:54:30,603 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46685] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 16:54:30,608 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/ea29850f58e32db8bff448bce6d2160a/29=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/30=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/31=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/32=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/33=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/34=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/35=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/36=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/37=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/38=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/39=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/40=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/41=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/42=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/43=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/44=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/45=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/46=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/47=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/48=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/49=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/50=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/51=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/52=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/53=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/54=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/55=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/56=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/57=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/58=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/59=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/60=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/61=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/62=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/63=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/64=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/65=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/66=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/67=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/68=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/69=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/70=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/71=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/72=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/73=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/74=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/75=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/76=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/77=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/78=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/79=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/80=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/81=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/82=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1.1689872067118, lastWalPosition=8659, nbRowKeys=54, nbHFiles=0, heapSize=11232, lastSeqIds={}, endOfFile=false,usedBufferSize=11232] 2023-07-20 16:54:30,615 DEBUG [SinkThread-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 16:54:30,617 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34208, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-07-20 16:54:30,619 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45821] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 16:54:30,620 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45821] zookeeper.ReadOnlyZKClient(139): Connect 0x1f683fbd to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:30,628 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45821] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@59335055, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:30,629 DEBUG [hconnection-0x2ebda7f0-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:30,632 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55568, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-07-20 16:54:30,646 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45821] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 16:54:30,655 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:30,690 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:30,695 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 2989 2023-07-20 16:54:30,695 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 25 WAL entries eligible for replication 2023-07-20 16:54:30,696 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 2989 2023-07-20 16:54:30,696 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/2627e10c9cf6db0694d7b55799cf6705/4=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/5=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/6=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/7=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/8=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/9=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/10=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/11=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/12=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/13=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/14=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/15=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/16=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/17=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/18=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/19=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/20=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/21=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/22=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/23=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/24=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/25=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/26=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/27=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/28=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0.1689872062821, lastWalPosition=2989, nbRowKeys=25, nbHFiles=0, heapSize=5200, lastSeqIds={}, endOfFile=false,usedBufferSize=5200] 2023-07-20 16:54:30,698 DEBUG [SinkThread-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 16:54:30,703 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57542, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-07-20 16:54:30,709 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43901] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 16:54:30,709 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43901] zookeeper.ReadOnlyZKClient(139): Connect 0x2102e4fe to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:30,717 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43901] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@2fc51eae, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:30,720 DEBUG [hconnection-0x2bfa8f8e-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:30,720 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:30,720 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 21 WAL entries eligible for replication 2023-07-20 16:54:30,720 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:30,720 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/ea29850f58e32db8bff448bce6d2160a/83=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/84=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/85=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/86=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/87=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/88=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/89=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/90=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/91=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/92=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/93=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/94=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/95=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/96=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/97=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/98=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/99=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/100=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/101=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/102=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/103=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1.1689872067118, lastWalPosition=10975, nbRowKeys=21, nbHFiles=0, heapSize=4368, lastSeqIds={}, endOfFile=false,usedBufferSize=4368] 2023-07-20 16:54:30,721 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55584, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-20 16:54:30,724 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45821] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 16:54:30,734 DEBUG [hconnection-0x2bfa8f8e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:30,737 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57556, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-07-20 16:54:30,740 INFO [jenkins-hbase4:43901Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:54:30,740 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45821] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 16:54:30,744 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=43901] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 16:54:30,746 INFO [jenkins-hbase4:46685Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:54:30,746 INFO [jenkins-hbase4:46685Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 25 2023-07-20 16:54:30,750 INFO [jenkins-hbase4:45493Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:54:30,751 INFO [jenkins-hbase4:45821Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:54:30,751 INFO [jenkins-hbase4:45821Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 75 2023-07-20 16:54:30,767 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:30,800 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:30,815 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 8869 2023-07-20 16:54:30,815 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 56 WAL entries eligible for replication 2023-07-20 16:54:30,815 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 8869 2023-07-20 16:54:30,816 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/2627e10c9cf6db0694d7b55799cf6705/29=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/30=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/31=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/32=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/33=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/34=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/35=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/36=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/37=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/38=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/39=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/40=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/41=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/42=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/43=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/44=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/45=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/46=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/47=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/48=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/49=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/50=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/51=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/52=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/53=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/54=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/55=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/56=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/57=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/58=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/59=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/60=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/61=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/62=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/63=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/64=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/65=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/66=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/67=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/68=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/69=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/70=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/71=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/72=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/73=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/74=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/75=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/76=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/77=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/78=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/79=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/80=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/81=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/82=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/83=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/84=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0.1689872062821, lastWalPosition=8869, nbRowKeys=56, nbHFiles=0, heapSize=11648, lastSeqIds={}, endOfFile=false,usedBufferSize=11648] 2023-07-20 16:54:30,819 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45821] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 16:54:30,828 DEBUG [hconnection-0x2ebda7f0-shared-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:30,838 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57570, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-07-20 16:54:30,840 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:30,843 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45821] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 16:54:30,876 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:30,935 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:30,949 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 16348 2023-07-20 16:54:30,950 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 70 WAL entries eligible for replication 2023-07-20 16:54:30,950 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 16348 2023-07-20 16:54:30,951 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/2627e10c9cf6db0694d7b55799cf6705/85=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/86=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/87=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/88=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/89=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/90=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/91=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/92=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/93=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/94=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/95=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/96=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/97=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/98=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/99=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/100=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/101=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/102=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/103=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/104=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/105=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/106=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/107=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/108=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/109=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/110=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/111=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/112=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/113=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/114=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/115=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/116=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/117=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/118=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/119=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/120=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/121=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/122=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/123=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/124=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/125=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/126=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/127=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/128=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/129=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/130=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/131=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/132=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/133=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/134=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/135=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/136=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/137=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/138=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/139=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/140=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/141=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/142=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/143=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/144=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/145=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/146=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/147=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/148=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/149=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/150=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/151=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/152=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/153=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/154=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0.1689872062821, lastWalPosition=16348, nbRowKeys=70, nbHFiles=0, heapSize=14560, lastSeqIds={}, endOfFile=false,usedBufferSize=14560] 2023-07-20 16:54:30,952 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:30,953 DEBUG [SinkThread-5] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 16:54:30,957 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55598, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=AdminService 2023-07-20 16:54:30,960 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45493] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 16:54:30,960 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45493] zookeeper.ReadOnlyZKClient(139): Connect 0x3d30e940 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:30,972 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45493] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@33f70096, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:30,974 DEBUG [hconnection-0x37c88230-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:30,978 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55612, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-20 16:54:30,990 DEBUG [hconnection-0x37c88230-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:30,993 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57574, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=ClientService 2023-07-20 16:54:30,994 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:30,998 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45493] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 16:54:31,044 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:31,065 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:31,070 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:31,070 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 49 WAL entries eligible for replication 2023-07-20 16:54:31,070 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:31,071 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/2627e10c9cf6db0694d7b55799cf6705/155=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/156=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/157=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/158=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/159=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/160=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/161=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/162=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/163=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/164=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/165=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/166=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/167=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/168=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/169=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/170=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/171=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/172=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/173=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/174=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/175=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/176=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/177=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/178=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/179=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/180=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/181=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/182=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/183=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/184=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/185=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/186=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/187=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/188=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/189=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/190=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/191=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/192=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/193=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/194=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/195=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/196=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/197=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/198=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/199=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/200=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/201=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/202=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/203=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0.1689872062821, lastWalPosition=21752, nbRowKeys=49, nbHFiles=0, heapSize=10192, lastSeqIds={}, endOfFile=false,usedBufferSize=10192] 2023-07-20 16:54:31,074 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45821] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 16:54:31,078 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45821] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 16:54:31,104 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:31,154 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:31,179 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:31,183 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:31,211 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:31,262 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:31,290 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:31,292 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:31,319 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:31,370 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:31,408 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:31,409 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:31,424 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:31,477 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:31,515 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:31,515 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:31,530 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:31,536 DEBUG [hconnection-0x137134cf-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:31,539 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55626, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:54:31,559 DEBUG [hconnection-0x137134cf-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:31,563 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57590, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:54:31,568 DEBUG [Listener at localhost/33489] replication.TestReplicationSyncUpTool(145): mimicSyncUpAfterDelete 2023-07-20 16:54:31,568 INFO [Listener at localhost/33489] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-20 16:54:31,568 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e1a2d76 to 127.0.0.1:56032 2023-07-20 16:54:31,568 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:31,569 DEBUG [Listener at localhost/33489] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-20 16:54:31,569 DEBUG [Listener at localhost/33489] util.JVMClusterUtil(257): Found active master hash=1539507352, stopped=false 2023-07-20 16:54:31,569 INFO [Listener at localhost/33489] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,45867,1689872065106 2023-07-20 16:54:31,573 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 16:54:31,573 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 16:54:31,573 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 16:54:31,573 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 16:54:31,573 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:54:31,573 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 16:54:31,573 INFO [Listener at localhost/33489] procedure2.ProcedureExecutor(629): Stopping 2023-07-20 16:54:31,575 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7c50b214 to 127.0.0.1:56032 2023-07-20 16:54:31,575 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:31,576 INFO [Listener at localhost/33489] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45493,1689872065148' ***** 2023-07-20 16:54:31,576 INFO [Listener at localhost/33489] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 16:54:31,576 INFO [Listener at localhost/33489] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46685,1689872065178' ***** 2023-07-20 16:54:31,576 INFO [Listener at localhost/33489] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 16:54:31,576 INFO [Listener at localhost/33489] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43901,1689872065207' ***** 2023-07-20 16:54:31,576 INFO [Listener at localhost/33489] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 16:54:31,576 INFO [Listener at localhost/33489] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45821,1689872065238' ***** 2023-07-20 16:54:31,576 INFO [Listener at localhost/33489] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 16:54:31,577 INFO [RS:0;jenkins-hbase4:45493] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 16:54:31,577 INFO [RS:0;jenkins-hbase4:45493] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 16:54:31,578 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 16:54:31,578 INFO [RS:3;jenkins-hbase4:45821] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 16:54:31,580 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 16:54:31,578 INFO [RS:2;jenkins-hbase4:43901] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 16:54:31,577 INFO [RS:1;jenkins-hbase4:46685] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 16:54:31,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:54:31,580 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 16:54:31,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:54:31,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:54:31,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:54:31,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:54:31,580 INFO [RS:1;jenkins-hbase4:46685] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 16:54:31,580 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 16:54:31,580 INFO [RS:2;jenkins-hbase4:43901] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 16:54:31,580 INFO [RS:3;jenkins-hbase4:45821] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 16:54:31,579 INFO [RS:0;jenkins-hbase4:45493] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 16:54:31,584 INFO [RS:3;jenkins-hbase4:45821] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 16:54:31,585 INFO [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(3305): Received CLOSE for 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:31,583 INFO [RS:2;jenkins-hbase4:43901] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 16:54:31,583 INFO [RS:1;jenkins-hbase4:46685] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 16:54:31,585 INFO [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(3305): Received CLOSE for c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:31,585 INFO [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(3305): Received CLOSE for 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:31,585 INFO [RS:1;jenkins-hbase4:46685] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:31,585 DEBUG [RS:1;jenkins-hbase4:46685] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x17376923 to 127.0.0.1:56032 2023-07-20 16:54:31,585 DEBUG [RS:1;jenkins-hbase4:46685] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:31,586 INFO [RS:1;jenkins-hbase4:46685] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46685,1689872065178; all regions closed. 2023-07-20 16:54:31,589 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:31,595 INFO [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:31,595 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:31,595 DEBUG [RS:3;jenkins-hbase4:45821] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x62bfb661 to 127.0.0.1:56032 2023-07-20 16:54:31,595 INFO [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:31,595 DEBUG [RS:3;jenkins-hbase4:45821] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:31,595 INFO [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:31,595 DEBUG [RS:2;jenkins-hbase4:43901] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0cd00850 to 127.0.0.1:56032 2023-07-20 16:54:31,598 DEBUG [RS:2;jenkins-hbase4:43901] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:31,595 DEBUG [RS:0;jenkins-hbase4:45493] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7c6aa926 to 127.0.0.1:56032 2023-07-20 16:54:31,600 DEBUG [RS:0;jenkins-hbase4:45493] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:31,600 INFO [RS:0;jenkins-hbase4:45493] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 16:54:31,600 INFO [RS:0;jenkins-hbase4:45493] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 16:54:31,600 INFO [RS:0;jenkins-hbase4:45493] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 16:54:31,600 INFO [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-20 16:54:31,600 INFO [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-20 16:54:31,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c96c52d963776c1249f8d28dc1cee95e, disabling compactions & flushes 2023-07-20 16:54:31,600 INFO [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-20 16:54:31,600 DEBUG [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, 058a0bfd9d8bf39d8c91089a151cb277=t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.} 2023-07-20 16:54:31,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 058a0bfd9d8bf39d8c91089a151cb277, disabling compactions & flushes 2023-07-20 16:54:31,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7ebeef24b783c4189ed905a5659b7e20, disabling compactions & flushes 2023-07-20 16:54:31,595 INFO [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-20 16:54:31,602 DEBUG [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(1478): Online Regions={7ebeef24b783c4189ed905a5659b7e20=hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.} 2023-07-20 16:54:31,601 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:31,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:31,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:31,600 DEBUG [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(1478): Online Regions={c96c52d963776c1249f8d28dc1cee95e=t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.} 2023-07-20 16:54:31,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:31,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. after waiting 0 ms 2023-07-20 16:54:31,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:31,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. after waiting 0 ms 2023-07-20 16:54:31,603 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:31,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:31,604 DEBUG [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(1504): Waiting on 058a0bfd9d8bf39d8c91089a151cb277, 1588230740 2023-07-20 16:54:31,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:31,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. after waiting 0 ms 2023-07-20 16:54:31,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:31,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 7ebeef24b783c4189ed905a5659b7e20 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-20 16:54:31,606 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-20 16:54:31,605 DEBUG [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(1504): Waiting on c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:31,605 DEBUG [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(1504): Waiting on 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:31,606 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-20 16:54:31,606 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-20 16:54:31,606 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-20 16:54:31,606 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-20 16:54:31,607 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.65 KB heapSize=7.06 KB 2023-07-20 16:54:31,606 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 058a0bfd9d8bf39d8c91089a151cb277 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-20 16:54:31,606 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing c96c52d963776c1249f8d28dc1cee95e 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-20 16:54:31,610 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:54:31,614 DEBUG [RS:1;jenkins-hbase4:46685] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs 2023-07-20 16:54:31,614 INFO [RS:1;jenkins-hbase4:46685] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C46685%2C1689872065178.jenkins-hbase4.apache.org%2C46685%2C1689872065178.regiongroup-0:(num 1689872065857) 2023-07-20 16:54:31,614 DEBUG [RS:1;jenkins-hbase4:46685] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:31,614 INFO [RS:1;jenkins-hbase4:46685] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:54:31,614 INFO [RS:1;jenkins-hbase4:46685] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-20 16:54:31,615 INFO [RS:1;jenkins-hbase4:46685] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 16:54:31,615 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 16:54:31,615 INFO [RS:1;jenkins-hbase4:46685] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 16:54:31,615 INFO [RS:1;jenkins-hbase4:46685] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 16:54:31,615 DEBUG [RS:1;jenkins-hbase4:46685] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x52bed061 to 127.0.0.1:56032 2023-07-20 16:54:31,615 DEBUG [RS:1;jenkins-hbase4:46685] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:31,617 INFO [RS:1;jenkins-hbase4:46685] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46685 2023-07-20 16:54:31,633 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:54:31,633 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:31,635 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:54:31,635 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:54:31,637 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:31,637 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:54:31,637 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:31,637 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:54:31,637 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:31,637 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:54:31,637 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,46685,1689872065178 2023-07-20 16:54:31,637 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310017, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:54:31,637 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:31,637 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:54:31,637 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:54:31,639 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 16:54:31,641 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46685,1689872065178] 2023-07-20 16:54:31,641 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46685,1689872065178; numProcessing=1 2023-07-20 16:54:31,642 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:31,643 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46685,1689872065178 already deleted, retry=false 2023-07-20 16:54:31,643 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46685,1689872065178 expired; onlineServers=3 2023-07-20 16:54:31,696 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:31,709 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/.tmp/info/ab70d834ddd448fe9bb282fb38cf6ad4 2023-07-20 16:54:31,709 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/.tmp/info/730b9259ce88426992f0c3d8aad85399 2023-07-20 16:54:31,710 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=8 (bloomFilter=true), to=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/.tmp/cf1/6419e14a3bbe44399930ef7d7a7c0e8b 2023-07-20 16:54:31,717 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/.tmp/cf1/806acc8751334bdeaac05f420a6fcdb3 2023-07-20 16:54:31,728 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-20 16:54:31,728 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-20 16:54:31,743 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:31,749 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:31,750 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:31,753 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-20 16:54:31,753 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-20 16:54:31,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/.tmp/cf1/6419e14a3bbe44399930ef7d7a7c0e8b as hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1/6419e14a3bbe44399930ef7d7a7c0e8b 2023-07-20 16:54:31,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/.tmp/info/ab70d834ddd448fe9bb282fb38cf6ad4 as hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/info/ab70d834ddd448fe9bb282fb38cf6ad4 2023-07-20 16:54:31,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/.tmp/cf1/806acc8751334bdeaac05f420a6fcdb3 as hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1/806acc8751334bdeaac05f420a6fcdb3 2023-07-20 16:54:31,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/info/ab70d834ddd448fe9bb282fb38cf6ad4, entries=2, sequenceid=6, filesize=4.8 K 2023-07-20 16:54:31,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1/806acc8751334bdeaac05f420a6fcdb3, entries=100, sequenceid=7, filesize=8.3 K 2023-07-20 16:54:31,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1/6419e14a3bbe44399930ef7d7a7c0e8b, entries=200, sequenceid=8, filesize=12.2 K 2023-07-20 16:54:31,775 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:31,775 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:46685-0x10183a7c1310009, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:31,775 INFO [RS:1;jenkins-hbase4:46685] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46685,1689872065178; zookeeper connection closed. 2023-07-20 16:54:31,777 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7d138c2] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7d138c2 2023-07-20 16:54:31,780 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 7ebeef24b783c4189ed905a5659b7e20 in 175ms, sequenceid=6, compaction requested=false 2023-07-20 16:54:31,780 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for c96c52d963776c1249f8d28dc1cee95e in 175ms, sequenceid=8, compaction requested=false 2023-07-20 16:54:31,780 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 058a0bfd9d8bf39d8c91089a151cb277 in 175ms, sequenceid=7, compaction requested=false 2023-07-20 16:54:31,789 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-20 16:54:31,792 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-20 16:54:31,796 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/.tmp/table/1619f7b97a8e4850aae967c3b3d7b384 2023-07-20 16:54:31,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-07-20 16:54:31,800 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:31,800 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 058a0bfd9d8bf39d8c91089a151cb277: 2023-07-20 16:54:31,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:31,804 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:31,805 DEBUG [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-20 16:54:31,805 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-20 16:54:31,806 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/recovered.edits/11.seqid, newMaxSeqId=11, maxSeqId=1 2023-07-20 16:54:31,806 DEBUG [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(1504): Waiting on c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:31,806 DEBUG [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(1504): Waiting on 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:31,807 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:31,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7ebeef24b783c4189ed905a5659b7e20: 2023-07-20 16:54:31,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:31,808 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/.tmp/info/730b9259ce88426992f0c3d8aad85399 as hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info/730b9259ce88426992f0c3d8aad85399 2023-07-20 16:54:31,808 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:31,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c96c52d963776c1249f8d28dc1cee95e: 2023-07-20 16:54:31,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:31,815 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info/730b9259ce88426992f0c3d8aad85399, entries=30, sequenceid=19, filesize=8.1 K 2023-07-20 16:54:31,816 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/.tmp/table/1619f7b97a8e4850aae967c3b3d7b384 as hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/table/1619f7b97a8e4850aae967c3b3d7b384 2023-07-20 16:54:31,822 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/table/1619f7b97a8e4850aae967c3b3d7b384, entries=6, sequenceid=19, filesize=4.8 K 2023-07-20 16:54:31,823 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.65 KB/3738, heapSize ~6.78 KB/6944, currentSize=0 B/0 for 1588230740 in 217ms, sequenceid=19, compaction requested=false 2023-07-20 16:54:31,834 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-20 16:54:31,835 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-20 16:54:31,835 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-20 16:54:31,835 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-20 16:54:31,836 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-20 16:54:31,849 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:31,855 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:31,856 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:31,910 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:31,955 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:31,961 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:31,962 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:32,005 INFO [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45493,1689872065148; all regions closed. 2023-07-20 16:54:32,005 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:32,007 INFO [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43901,1689872065207; all regions closed. 2023-07-20 16:54:32,008 INFO [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45821,1689872065238; all regions closed. 2023-07-20 16:54:32,008 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:32,011 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:32,031 DEBUG [RS:2;jenkins-hbase4:43901] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs 2023-07-20 16:54:32,031 INFO [RS:2;jenkins-hbase4:43901] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C43901%2C1689872065207.jenkins-hbase4.apache.org%2C43901%2C1689872065207.regiongroup-1:(num 1689872069483) 2023-07-20 16:54:32,031 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:32,034 DEBUG [RS:3;jenkins-hbase4:45821] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs 2023-07-20 16:54:32,034 INFO [RS:3;jenkins-hbase4:45821] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C45821%2C1689872065238.jenkins-hbase4.apache.org%2C45821%2C1689872065238.regiongroup-1:(num 1689872066465) 2023-07-20 16:54:32,034 DEBUG [RS:0;jenkins-hbase4:45493] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs 2023-07-20 16:54:32,034 INFO [RS:0;jenkins-hbase4:45493] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C45493%2C1689872065148.meta:.meta(num 1689872065974) 2023-07-20 16:54:32,034 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:32,036 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:32,050 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:32,052 DEBUG [RS:2;jenkins-hbase4:43901] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs 2023-07-20 16:54:32,052 INFO [RS:2;jenkins-hbase4:43901] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C43901%2C1689872065207.jenkins-hbase4.apache.org%2C43901%2C1689872065207.regiongroup-0:(num 1689872065849) 2023-07-20 16:54:32,053 DEBUG [RS:2;jenkins-hbase4:43901] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:32,053 INFO [RS:2;jenkins-hbase4:43901] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:54:32,053 INFO [RS:2;jenkins-hbase4:43901] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-20 16:54:32,054 INFO [RS:2;jenkins-hbase4:43901] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 16:54:32,054 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 16:54:32,054 INFO [RS:2;jenkins-hbase4:43901] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 16:54:32,054 INFO [RS:2;jenkins-hbase4:43901] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 16:54:32,054 DEBUG [RS:2;jenkins-hbase4:43901] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2102e4fe to 127.0.0.1:56032 2023-07-20 16:54:32,054 DEBUG [RS:2;jenkins-hbase4:43901] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:32,055 INFO [RS:2;jenkins-hbase4:43901] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43901 2023-07-20 16:54:32,065 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:32,065 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310017, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:54:32,065 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:54:32,065 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:32,065 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 16:54:32,066 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,43901,1689872065207 2023-07-20 16:54:32,069 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:32,070 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43901,1689872065207] 2023-07-20 16:54:32,070 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43901,1689872065207; numProcessing=2 2023-07-20 16:54:32,070 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:32,070 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:32,071 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,43901,1689872065207 already deleted, retry=false 2023-07-20 16:54:32,071 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43901,1689872065207 expired; onlineServers=2 2023-07-20 16:54:32,156 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:32,177 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:32,178 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:32,178 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:32,264 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:32,283 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:32,283 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:32,284 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:32,371 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:32,390 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:32,390 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:32,390 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:32,453 DEBUG [RS:3;jenkins-hbase4:45821] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs 2023-07-20 16:54:32,454 DEBUG [RS:0;jenkins-hbase4:45493] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs 2023-07-20 16:54:32,454 INFO [RS:3;jenkins-hbase4:45821] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C45821%2C1689872065238.jenkins-hbase4.apache.org%2C45821%2C1689872065238.regiongroup-0:(num 1689872065887) 2023-07-20 16:54:32,454 INFO [RS:0;jenkins-hbase4:45493] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C45493%2C1689872065148.jenkins-hbase4.apache.org%2C45493%2C1689872065148.regiongroup-0:(num 1689872065870) 2023-07-20 16:54:32,454 DEBUG [RS:3;jenkins-hbase4:45821] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:32,454 INFO [RS:3;jenkins-hbase4:45821] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:54:32,454 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:32,454 INFO [RS:3;jenkins-hbase4:45821] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-20 16:54:32,455 INFO [RS:3;jenkins-hbase4:45821] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 16:54:32,455 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 16:54:32,455 INFO [RS:3;jenkins-hbase4:45821] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 16:54:32,455 INFO [RS:3;jenkins-hbase4:45821] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 16:54:32,455 DEBUG [RS:3;jenkins-hbase4:45821] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1f683fbd to 127.0.0.1:56032 2023-07-20 16:54:32,455 DEBUG [RS:3;jenkins-hbase4:45821] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:32,457 INFO [RS:3;jenkins-hbase4:45821] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45821 2023-07-20 16:54:32,461 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:32,462 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:54:32,462 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310017, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:54:32,463 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 16:54:32,462 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,45821,1689872065238 2023-07-20 16:54:32,465 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45821,1689872065238] 2023-07-20 16:54:32,465 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45821,1689872065238; numProcessing=3 2023-07-20 16:54:32,466 DEBUG [RS:0;jenkins-hbase4:45493] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs 2023-07-20 16:54:32,466 INFO [RS:0;jenkins-hbase4:45493] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C45493%2C1689872065148.jenkins-hbase4.apache.org%2C45493%2C1689872065148.regiongroup-1:(num 1689872068393) 2023-07-20 16:54:32,466 DEBUG [RS:0;jenkins-hbase4:45493] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:32,466 INFO [RS:0;jenkins-hbase4:45493] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:54:32,466 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,45821,1689872065238 already deleted, retry=false 2023-07-20 16:54:32,466 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45821,1689872065238 expired; onlineServers=1 2023-07-20 16:54:32,466 INFO [RS:0;jenkins-hbase4:45493] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-20 16:54:32,466 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 16:54:32,467 DEBUG [RS:0;jenkins-hbase4:45493] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3d30e940 to 127.0.0.1:56032 2023-07-20 16:54:32,467 DEBUG [RS:0;jenkins-hbase4:45493] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:32,468 INFO [RS:0;jenkins-hbase4:45493] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45493 2023-07-20 16:54:32,474 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,45493,1689872065148 2023-07-20 16:54:32,474 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:54:32,474 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310017, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:54:32,474 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 16:54:32,476 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:32,476 INFO [RS:2;jenkins-hbase4:43901] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43901,1689872065207; zookeeper connection closed. 2023-07-20 16:54:32,476 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45493,1689872065148] 2023-07-20 16:54:32,476 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:43901-0x10183a7c131000a, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:32,476 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45493,1689872065148; numProcessing=4 2023-07-20 16:54:32,477 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@468582d4] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@468582d4 2023-07-20 16:54:32,479 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:32,479 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,45493,1689872065148 already deleted, retry=false 2023-07-20 16:54:32,479 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45493,1689872065148 expired; onlineServers=0 2023-07-20 16:54:32,479 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45867,1689872065106' ***** 2023-07-20 16:54:32,479 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-20 16:54:32,480 DEBUG [M:0;jenkins-hbase4:45867] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ffc9b38, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 16:54:32,481 INFO [M:0;jenkins-hbase4:45867] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45867,1689872065106 2023-07-20 16:54:32,481 INFO [M:0;jenkins-hbase4:45867] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45867,1689872065106; all regions closed. 2023-07-20 16:54:32,481 DEBUG [M:0;jenkins-hbase4:45867] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:32,481 DEBUG [M:0;jenkins-hbase4:45867] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-20 16:54:32,481 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-20 16:54:32,481 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872065500] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872065500,5,FailOnTimeoutGroup] 2023-07-20 16:54:32,481 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872065502] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872065502,5,FailOnTimeoutGroup] 2023-07-20 16:54:32,481 DEBUG [M:0;jenkins-hbase4:45867] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-20 16:54:32,483 INFO [M:0;jenkins-hbase4:45867] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-20 16:54:32,483 INFO [M:0;jenkins-hbase4:45867] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-20 16:54:32,483 INFO [M:0;jenkins-hbase4:45867] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-20 16:54:32,483 DEBUG [M:0;jenkins-hbase4:45867] master.HMaster(1512): Stopping service threads 2023-07-20 16:54:32,483 INFO [M:0;jenkins-hbase4:45867] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-20 16:54:32,485 ERROR [M:0;jenkins-hbase4:45867] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-07-20 16:54:32,485 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-20 16:54:32,485 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:54:32,485 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 16:54:32,485 INFO [M:0;jenkins-hbase4:45867] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-20 16:54:32,485 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-20 16:54:32,486 DEBUG [M:0;jenkins-hbase4:45867] zookeeper.ZKUtil(398): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-20 16:54:32,486 WARN [M:0;jenkins-hbase4:45867] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-20 16:54:32,486 INFO [M:0;jenkins-hbase4:45867] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-20 16:54:32,487 INFO [M:0;jenkins-hbase4:45867] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-20 16:54:32,488 DEBUG [M:0;jenkins-hbase4:45867] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-20 16:54:32,488 INFO [M:0;jenkins-hbase4:45867] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:32,488 DEBUG [M:0;jenkins-hbase4:45867] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:32,488 DEBUG [M:0;jenkins-hbase4:45867] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-20 16:54:32,488 DEBUG [M:0;jenkins-hbase4:45867] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:32,488 INFO [M:0;jenkins-hbase4:45867] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=55.40 KB heapSize=65.18 KB 2023-07-20 16:54:32,496 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:32,497 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:32,500 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:32,507 INFO [M:0;jenkins-hbase4:45867] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=55.40 KB at sequenceid=116 (bloomFilter=true), to=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/56183e65ade64a5ab9377e2784bcf14d 2023-07-20 16:54:32,513 DEBUG [M:0;jenkins-hbase4:45867] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/56183e65ade64a5ab9377e2784bcf14d as hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/56183e65ade64a5ab9377e2784bcf14d 2023-07-20 16:54:32,518 INFO [M:0;jenkins-hbase4:45867] regionserver.HStore(1080): Added hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/56183e65ade64a5ab9377e2784bcf14d, entries=14, sequenceid=116, filesize=8.1 K 2023-07-20 16:54:32,519 INFO [M:0;jenkins-hbase4:45867] regionserver.HRegion(2948): Finished flush of dataSize ~55.40 KB/56732, heapSize ~65.16 KB/66728, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 31ms, sequenceid=116, compaction requested=false 2023-07-20 16:54:32,523 INFO [M:0;jenkins-hbase4:45867] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:32,523 DEBUG [M:0;jenkins-hbase4:45867] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 16:54:32,523 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,45867,1689872065106 2023-07-20 16:54:32,529 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 16:54:32,529 INFO [M:0;jenkins-hbase4:45867] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-20 16:54:32,530 INFO [M:0;jenkins-hbase4:45867] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45867 2023-07-20 16:54:32,532 DEBUG [M:0;jenkins-hbase4:45867] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,45867,1689872065106 already deleted, retry=false 2023-07-20 16:54:32,585 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:32,659 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:32,659 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:32,661 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:32,676 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:32,676 INFO [M:0;jenkins-hbase4:45867] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45867,1689872065106; zookeeper connection closed. 2023-07-20 16:54:32,677 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:45867-0x10183a7c1310007, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:32,690 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:32,693 INFO [jenkins-hbase4:33103Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 300, current progress: walGroup [jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1]: currently replicating from: hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1.1689872067118 at position: 10975 walGroup [jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0]: currently replicating from: hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0.1689872062821 at position: 21752 2023-07-20 16:54:32,693 INFO [jenkins-hbase4:34183Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0]: currently replicating from: hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357/jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0.1689872062821 at position: -1 walGroup [jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1]: currently replicating from: hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357/jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1.1689872063702 at position: 600 2023-07-20 16:54:32,765 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:32,768 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:32,769 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:32,777 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:32,777 INFO [RS:0;jenkins-hbase4:45493] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45493,1689872065148; zookeeper connection closed. 2023-07-20 16:54:32,777 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45493-0x10183a7c1310008, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:32,778 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7d9fe324] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7d9fe324 2023-07-20 16:54:32,796 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:32,871 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:32,874 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:32,875 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:32,902 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:32,977 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:32,977 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:45821-0x10183a7c131000b, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:32,977 INFO [RS:3;jenkins-hbase4:45821] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45821,1689872065238; zookeeper connection closed. 2023-07-20 16:54:32,979 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@77e40768] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@77e40768 2023-07-20 16:54:32,979 INFO [Listener at localhost/33489] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 4 regionserver(s) complete 2023-07-20 16:54:32,981 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 21752 2023-07-20 16:54:32,987 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:32,990 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 10975 2023-07-20 16:54:33,008 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 600 2023-07-20 16:54:33,014 INFO [Listener at localhost/33489] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-20 16:54:33,014 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3448932c to 127.0.0.1:56032 2023-07-20 16:54:33,014 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:33,015 DEBUG [Listener at localhost/33489] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-20 16:54:33,015 DEBUG [Listener at localhost/33489] util.JVMClusterUtil(257): Found active master hash=1829693025, stopped=false 2023-07-20 16:54:33,015 INFO [Listener at localhost/33489] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,39639,1689872060057 2023-07-20 16:54:33,017 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:33103-0x10183a7c1310001, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-20 16:54:33,017 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-20 16:54:33,017 INFO [Listener at localhost/33489] procedure2.ProcedureExecutor(629): Stopping 2023-07-20 16:54:33,017 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-20 16:54:33,017 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2531dee3 to 127.0.0.1:56032 2023-07-20 16:54:33,017 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:54:33,019 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 16:54:33,018 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:33,020 INFO [Listener at localhost/33489] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33103,1689872061305' ***** 2023-07-20 16:54:33,020 INFO [Listener at localhost/33489] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 16:54:33,020 INFO [Listener at localhost/33489] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34183,1689872061357' ***** 2023-07-20 16:54:33,020 INFO [Listener at localhost/33489] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 16:54:33,020 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 16:54:33,021 INFO [RS:1;jenkins-hbase4:34183] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 16:54:33,021 INFO [RS:0;jenkins-hbase4:33103] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 16:54:33,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33103-0x10183a7c1310001, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 16:54:33,021 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 16:54:33,021 INFO [RS:1;jenkins-hbase4:34183] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 16:54:33,021 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 16:54:33,021 INFO [RS:0;jenkins-hbase4:33103] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 16:54:33,023 INFO [RS:0;jenkins-hbase4:33103] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 16:54:33,023 INFO [RS:1;jenkins-hbase4:34183] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 16:54:33,023 INFO [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(3305): Received CLOSE for 2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:54:33,024 INFO [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(3305): Received CLOSE for eec95f84b91ff86943540982735963f6 2023-07-20 16:54:33,024 INFO [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:33,024 DEBUG [RS:1;jenkins-hbase4:34183] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x03c82af4 to 127.0.0.1:56032 2023-07-20 16:54:33,024 DEBUG [RS:1;jenkins-hbase4:34183] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:33,024 INFO [RS:1;jenkins-hbase4:34183] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 16:54:33,024 INFO [RS:1;jenkins-hbase4:34183] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 16:54:33,024 INFO [RS:1;jenkins-hbase4:34183] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 16:54:33,024 INFO [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-20 16:54:33,027 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing eec95f84b91ff86943540982735963f6, disabling compactions & flushes 2023-07-20 16:54:33,027 INFO [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(3305): Received CLOSE for ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:54:33,027 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:54:33,027 INFO [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:33,027 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:54:33,027 DEBUG [RS:0;jenkins-hbase4:33103] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1290d046 to 127.0.0.1:56032 2023-07-20 16:54:33,027 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. after waiting 0 ms 2023-07-20 16:54:33,028 DEBUG [RS:0;jenkins-hbase4:33103] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:33,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:54:33,028 INFO [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-20 16:54:33,028 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing eec95f84b91ff86943540982735963f6 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-20 16:54:33,028 INFO [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-20 16:54:33,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2627e10c9cf6db0694d7b55799cf6705, disabling compactions & flushes 2023-07-20 16:54:33,028 DEBUG [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, eec95f84b91ff86943540982735963f6=hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6.} 2023-07-20 16:54:33,029 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-20 16:54:33,029 DEBUG [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(1504): Waiting on 1588230740, eec95f84b91ff86943540982735963f6 2023-07-20 16:54:33,029 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-20 16:54:33,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:54:33,030 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:54:33,030 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. after waiting 0 ms 2023-07-20 16:54:33,029 DEBUG [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(1478): Online Regions={2627e10c9cf6db0694d7b55799cf6705=t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705., ea29850f58e32db8bff448bce6d2160a=t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a.} 2023-07-20 16:54:33,030 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:54:33,030 DEBUG [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(1504): Waiting on 2627e10c9cf6db0694d7b55799cf6705, ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:54:33,029 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-20 16:54:33,030 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 2627e10c9cf6db0694d7b55799cf6705 2/2 column families, dataSize=12.70 KB heapSize=42.80 KB 2023-07-20 16:54:33,030 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-20 16:54:33,030 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-20 16:54:33,030 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.87 KB heapSize=7.44 KB 2023-07-20 16:54:33,063 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6/.tmp/info/3423509284054478986c16da9315751f 2023-07-20 16:54:33,072 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6/.tmp/info/3423509284054478986c16da9315751f as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6/info/3423509284054478986c16da9315751f 2023-07-20 16:54:33,075 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.74 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/.tmp/cf1/f822180116c0458bbb56c3e3d5669ccc 2023-07-20 16:54:33,082 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6/info/3423509284054478986c16da9315751f, entries=2, sequenceid=6, filesize=4.8 K 2023-07-20 16:54:33,085 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f822180116c0458bbb56c3e3d5669ccc 2023-07-20 16:54:33,086 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for eec95f84b91ff86943540982735963f6 in 58ms, sequenceid=6, compaction requested=false 2023-07-20 16:54:33,089 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:54:33,090 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-07-20 16:54:33,090 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-20 16:54:33,091 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/2627e10c9cf6db0694d7b55799cf6705/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0.1689872062821, lastWalPosition=28643, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=false,usedBufferSize=8120] 2023-07-20 16:54:33,091 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 28643 2023-07-20 16:54:33,092 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:54:33,105 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:33,105 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-07-20 16:54:33,106 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-20 16:54:33,106 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14321 2023-07-20 16:54:33,106 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/ea29850f58e32db8bff448bce6d2160a/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1.1689872067118, lastWalPosition=14321, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=false,usedBufferSize=4120] 2023-07-20 16:54:33,108 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-20 16:54:33,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:54:33,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for eec95f84b91ff86943540982735963f6: 2023-07-20 16:54:33,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:54:33,117 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-20 16:54:33,118 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 16:54:33,118 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-20 16:54:33,118 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357/jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1.1689872063702, lastWalPosition=1430, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 16:54:33,121 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.96 KB at sequenceid=206 (bloomFilter=true), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/.tmp/norep/685aeda030444458949aeb687cade474 2023-07-20 16:54:33,128 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 685aeda030444458949aeb687cade474 2023-07-20 16:54:33,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/.tmp/cf1/f822180116c0458bbb56c3e3d5669ccc as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/cf1/f822180116c0458bbb56c3e3d5669ccc 2023-07-20 16:54:33,140 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f822180116c0458bbb56c3e3d5669ccc 2023-07-20 16:54:33,140 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/cf1/f822180116c0458bbb56c3e3d5669ccc, entries=200, sequenceid=206, filesize=11.9 K 2023-07-20 16:54:33,142 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/.tmp/norep/685aeda030444458949aeb687cade474 as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/norep/685aeda030444458949aeb687cade474 2023-07-20 16:54:33,149 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 685aeda030444458949aeb687cade474 2023-07-20 16:54:33,149 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/norep/685aeda030444458949aeb687cade474, entries=101, sequenceid=206, filesize=8.3 K 2023-07-20 16:54:33,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~12.70 KB/13001, heapSize ~42.77 KB/43800, currentSize=0 B/0 for 2627e10c9cf6db0694d7b55799cf6705 in 120ms, sequenceid=206, compaction requested=false 2023-07-20 16:54:33,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/recovered.edits/209.seqid, newMaxSeqId=209, maxSeqId=1 2023-07-20 16:54:33,164 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:54:33,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2627e10c9cf6db0694d7b55799cf6705: 2023-07-20 16:54:33,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:54:33,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ea29850f58e32db8bff448bce6d2160a, disabling compactions & flushes 2023-07-20 16:54:33,165 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:54:33,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:54:33,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. after waiting 0 ms 2023-07-20 16:54:33,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:54:33,165 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ea29850f58e32db8bff448bce6d2160a 2/2 column families, dataSize=6.25 KB heapSize=21.71 KB 2023-07-20 16:54:33,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.76 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/.tmp/cf1/130b8e810ca245e0ab3ce35f11a7a777 2023-07-20 16:54:33,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 130b8e810ca245e0ab3ce35f11a7a777 2023-07-20 16:54:33,198 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-07-20 16:54:33,198 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 16:54:33,199 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-07-20 16:54:33,215 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:33,219 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.50 KB at sequenceid=106 (bloomFilter=true), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/.tmp/norep/e35537dcd33b41589483da406e2e9bf7 2023-07-20 16:54:33,222 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-07-20 16:54:33,222 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 16:54:33,222 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14562 2023-07-20 16:54:33,224 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-20 16:54:33,226 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e35537dcd33b41589483da406e2e9bf7 2023-07-20 16:54:33,227 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/.tmp/cf1/130b8e810ca245e0ab3ce35f11a7a777 as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/cf1/130b8e810ca245e0ab3ce35f11a7a777 2023-07-20 16:54:33,230 DEBUG [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-20 16:54:33,230 DEBUG [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(1504): Waiting on ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:54:33,234 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 130b8e810ca245e0ab3ce35f11a7a777 2023-07-20 16:54:33,234 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/cf1/130b8e810ca245e0ab3ce35f11a7a777, entries=100, sequenceid=106, filesize=8.2 K 2023-07-20 16:54:33,235 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/.tmp/norep/e35537dcd33b41589483da406e2e9bf7 as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/norep/e35537dcd33b41589483da406e2e9bf7 2023-07-20 16:54:33,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e35537dcd33b41589483da406e2e9bf7 2023-07-20 16:54:33,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/norep/e35537dcd33b41589483da406e2e9bf7, entries=51, sequenceid=106, filesize=6.5 K 2023-07-20 16:54:33,243 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~6.25 KB/6401, heapSize ~21.68 KB/22200, currentSize=0 B/0 for ea29850f58e32db8bff448bce6d2160a in 78ms, sequenceid=106, compaction requested=false 2023-07-20 16:54:33,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/recovered.edits/109.seqid, newMaxSeqId=109, maxSeqId=1 2023-07-20 16:54:33,255 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:54:33,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ea29850f58e32db8bff448bce6d2160a: 2023-07-20 16:54:33,256 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:54:33,305 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-07-20 16:54:33,321 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:33,328 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 15173 2023-07-20 16:54:33,328 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 16:54:33,330 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-20 16:54:33,412 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 29258 2023-07-20 16:54:33,426 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:33,430 DEBUG [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-20 16:54:33,430 INFO [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33103,1689872061305; all regions closed. 2023-07-20 16:54:33,431 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:33,437 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-20 16:54:33,438 DEBUG [RS:0;jenkins-hbase4:33103] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs 2023-07-20 16:54:33,438 INFO [RS:0;jenkins-hbase4:33103] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0:(num 1689872062821) 2023-07-20 16:54:33,438 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:33,444 DEBUG [RS:0;jenkins-hbase4:33103] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs 2023-07-20 16:54:33,444 INFO [RS:0;jenkins-hbase4:33103] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1:(num 1689872067118) 2023-07-20 16:54:33,444 DEBUG [RS:0;jenkins-hbase4:33103] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:33,444 INFO [RS:0;jenkins-hbase4:33103] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:54:33,445 INFO [RS:0;jenkins-hbase4:33103] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-20 16:54:33,445 INFO [RS:0;jenkins-hbase4:33103] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 16:54:33,445 INFO [RS:0;jenkins-hbase4:33103] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 16:54:33,445 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 16:54:33,445 INFO [RS:0;jenkins-hbase4:33103] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 16:54:33,445 INFO [RS:0;jenkins-hbase4:33103] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-20 16:54:33,486 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/.tmp/info/b7ee159ed6a148b298dac2473952132c 2023-07-20 16:54:33,503 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/.tmp/rep_barrier/9073089ba3a74d4a8bf812e10f2caad7 2023-07-20 16:54:33,513 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0.1689872062821 was moved to hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0.1689872062821 2023-07-20 16:54:33,520 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=258 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/.tmp/table/b49b27dd474648f69f5e68f6f5886f66 2023-07-20 16:54:33,527 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/.tmp/info/b7ee159ed6a148b298dac2473952132c as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/info/b7ee159ed6a148b298dac2473952132c 2023-07-20 16:54:33,531 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:33,533 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/info/b7ee159ed6a148b298dac2473952132c, entries=30, sequenceid=19, filesize=8.1 K 2023-07-20 16:54:33,534 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/.tmp/rep_barrier/9073089ba3a74d4a8bf812e10f2caad7 as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/rep_barrier/9073089ba3a74d4a8bf812e10f2caad7 2023-07-20 16:54:33,540 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/rep_barrier/9073089ba3a74d4a8bf812e10f2caad7, entries=2, sequenceid=19, filesize=4.9 K 2023-07-20 16:54:33,541 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/.tmp/table/b49b27dd474648f69f5e68f6f5886f66 as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/table/b49b27dd474648f69f5e68f6f5886f66 2023-07-20 16:54:33,542 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-20 16:54:33,547 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310017, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:33,547 DEBUG [RS:0;jenkins-hbase4:33103] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x67310515 to 127.0.0.1:56032 2023-07-20 16:54:33,547 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310017, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:33,547 DEBUG [RS:0;jenkins-hbase4:33103] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:33,549 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/table/b49b27dd474648f69f5e68f6f5886f66, entries=6, sequenceid=19, filesize=4.8 K 2023-07-20 16:54:33,550 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.87 KB/3964, heapSize ~7.39 KB/7568, currentSize=0 B/0 for 1588230740 in 520ms, sequenceid=19, compaction requested=false 2023-07-20 16:54:33,560 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=1 2023-07-20 16:54:33,561 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-20 16:54:33,561 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-20 16:54:33,562 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-20 16:54:33,562 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-20 16:54:33,630 INFO [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34183,1689872061357; all regions closed. 2023-07-20 16:54:33,630 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:33,637 DEBUG [RS:1;jenkins-hbase4:34183] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs 2023-07-20 16:54:33,637 INFO [RS:1;jenkins-hbase4:34183] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C34183%2C1689872061357.meta:.meta(num 1689872062936) 2023-07-20 16:54:33,638 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:33,638 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:54:33,647 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 1430 2023-07-20 16:54:33,649 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357/jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0.1689872062821 2023-07-20 16:54:33,649 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-20 16:54:33,649 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 16:54:33,650 DEBUG [RS:1;jenkins-hbase4:34183] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs 2023-07-20 16:54:33,650 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357/jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0.1689872062821, lastWalPosition=-1, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-07-20 16:54:33,650 INFO [RS:1;jenkins-hbase4:34183] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0:(num 1689872062821) 2023-07-20 16:54:33,650 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-20 16:54:33,650 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-20 16:54:33,651 INFO [RS:0;jenkins-hbase4:33103] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1 terminated 2023-07-20 16:54:33,651 INFO [RS:0;jenkins-hbase4:33103] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1 terminated 2023-07-20 16:54:33,652 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0.1689872062821] 2023-07-20 16:54:33,652 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:33,653 INFO [RS:0;jenkins-hbase4:33103] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33103 2023-07-20 16:54:33,659 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 16:54:33,659 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:33103-0x10183a7c1310001, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:33,659 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:33103-0x10183a7c1310001, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 16:54:33,659 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:33,659 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 16:54:33,660 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33103,1689872061305] 2023-07-20 16:54:33,660 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33103,1689872061305; numProcessing=1 2023-07-20 16:54:33,661 DEBUG [RS:1;jenkins-hbase4:34183] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs 2023-07-20 16:54:33,661 INFO [RS:1;jenkins-hbase4:34183] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1:(num 1689872063702) 2023-07-20 16:54:33,661 DEBUG [RS:1;jenkins-hbase4:34183] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:33,661 INFO [RS:1;jenkins-hbase4:34183] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:54:33,662 INFO [RS:1;jenkins-hbase4:34183] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-20 16:54:33,662 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 16:54:33,662 INFO [RS:1;jenkins-hbase4:34183] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-20 16:54:33,662 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,33103,1689872061305 already deleted, retry=false 2023-07-20 16:54:33,663 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33103,1689872061305 expired; onlineServers=1 2023-07-20 16:54:33,749 INFO [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357/jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1.1689872063702 was moved to hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs/jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1.1689872063702 2023-07-20 16:54:33,761 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:33103-0x10183a7c1310001, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:33,761 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:33103-0x10183a7c1310001, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:33,761 INFO [RS:0;jenkins-hbase4:33103] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33103,1689872061305; zookeeper connection closed. 2023-07-20 16:54:33,761 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@34e83bb0] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@34e83bb0 2023-07-20 16:54:33,861 DEBUG [RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310018, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:33,861 DEBUG [RS:1;jenkins-hbase4:34183] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x05efb24f to 127.0.0.1:56032 2023-07-20 16:54:33,861 DEBUG [RS:1;jenkins-hbase4:34183] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:33,861 INFO [RS:1;jenkins-hbase4:34183] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-0,1 terminated 2023-07-20 16:54:33,861 INFO [RS:1;jenkins-hbase4:34183] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/jenkins-hbase4:0-0.replicationSource,1.replicationSource.shipperjenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1 terminated 2023-07-20 16:54:33,863 INFO [RS:1;jenkins-hbase4:34183] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34183 2023-07-20 16:54:33,864 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:33,864 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 16:54:33,867 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34183,1689872061357] 2023-07-20 16:54:33,867 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34183,1689872061357; numProcessing=2 2023-07-20 16:54:33,868 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,34183,1689872061357 already deleted, retry=false 2023-07-20 16:54:33,868 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34183,1689872061357 expired; onlineServers=0 2023-07-20 16:54:33,868 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39639,1689872060057' ***** 2023-07-20 16:54:33,868 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-20 16:54:33,869 DEBUG [M:0;jenkins-hbase4:39639] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2c33ec5a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 16:54:33,869 INFO [M:0;jenkins-hbase4:39639] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39639,1689872060057 2023-07-20 16:54:33,869 INFO [M:0;jenkins-hbase4:39639] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39639,1689872060057; all regions closed. 2023-07-20 16:54:33,869 DEBUG [M:0;jenkins-hbase4:39639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:33,869 DEBUG [M:0;jenkins-hbase4:39639] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-20 16:54:33,869 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-20 16:54:33,869 DEBUG [M:0;jenkins-hbase4:39639] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-20 16:54:33,869 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872062382] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872062382,5,FailOnTimeoutGroup] 2023-07-20 16:54:33,870 INFO [M:0;jenkins-hbase4:39639] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-20 16:54:33,869 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872062380] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872062380,5,FailOnTimeoutGroup] 2023-07-20 16:54:33,871 INFO [M:0;jenkins-hbase4:39639] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-20 16:54:33,872 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-20 16:54:33,872 INFO [M:0;jenkins-hbase4:39639] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-20 16:54:33,872 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:54:33,872 DEBUG [M:0;jenkins-hbase4:39639] master.HMaster(1512): Stopping service threads 2023-07-20 16:54:33,872 INFO [M:0;jenkins-hbase4:39639] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-20 16:54:33,873 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-20 16:54:33,873 ERROR [M:0;jenkins-hbase4:39639] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] 2023-07-20 16:54:33,873 INFO [M:0;jenkins-hbase4:39639] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-20 16:54:33,873 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-20 16:54:33,873 DEBUG [M:0;jenkins-hbase4:39639] zookeeper.ZKUtil(398): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-20 16:54:33,873 WARN [M:0;jenkins-hbase4:39639] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-20 16:54:33,873 INFO [M:0;jenkins-hbase4:39639] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-20 16:54:33,874 INFO [M:0;jenkins-hbase4:39639] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-20 16:54:33,875 DEBUG [M:0;jenkins-hbase4:39639] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-20 16:54:33,875 INFO [M:0;jenkins-hbase4:39639] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:33,875 DEBUG [M:0;jenkins-hbase4:39639] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:33,875 DEBUG [M:0;jenkins-hbase4:39639] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-20 16:54:33,875 DEBUG [M:0;jenkins-hbase4:39639] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:33,875 INFO [M:0;jenkins-hbase4:39639] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=61.04 KB heapSize=72.60 KB 2023-07-20 16:54:33,885 INFO [M:0;jenkins-hbase4:39639] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=61.04 KB at sequenceid=136 (bloomFilter=true), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/34f5e65d0ce74ea1928444c9b7a76dc3 2023-07-20 16:54:33,891 DEBUG [M:0;jenkins-hbase4:39639] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/34f5e65d0ce74ea1928444c9b7a76dc3 as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/34f5e65d0ce74ea1928444c9b7a76dc3 2023-07-20 16:54:33,897 INFO [M:0;jenkins-hbase4:39639] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/34f5e65d0ce74ea1928444c9b7a76dc3, entries=17, sequenceid=136, filesize=8.5 K 2023-07-20 16:54:33,898 INFO [M:0;jenkins-hbase4:39639] regionserver.HRegion(2948): Finished flush of dataSize ~61.04 KB/62508, heapSize ~72.59 KB/74328, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 23ms, sequenceid=136, compaction requested=false 2023-07-20 16:54:33,900 INFO [M:0;jenkins-hbase4:39639] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:54:33,900 DEBUG [M:0;jenkins-hbase4:39639] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 16:54:33,900 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/WALs/jenkins-hbase4.apache.org,39639,1689872060057 2023-07-20 16:54:33,904 INFO [M:0;jenkins-hbase4:39639] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-20 16:54:33,904 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 16:54:33,905 INFO [M:0;jenkins-hbase4:39639] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39639 2023-07-20 16:54:33,907 DEBUG [M:0;jenkins-hbase4:39639] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,39639,1689872060057 already deleted, retry=false 2023-07-20 16:54:33,916 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 16:54:34,063 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:34,063 INFO [M:0;jenkins-hbase4:39639] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39639,1689872060057; zookeeper connection closed. 2023-07-20 16:54:34,064 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): master:39639-0x10183a7c1310000, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:34,164 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:34,164 INFO [RS:1;jenkins-hbase4:34183] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34183,1689872061357; zookeeper connection closed. 2023-07-20 16:54:34,164 DEBUG [Listener at localhost/36209-EventThread] zookeeper.ZKWatcher(600): regionserver:34183-0x10183a7c1310002, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:54:34,164 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4661e935] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4661e935 2023-07-20 16:54:34,164 INFO [Listener at localhost/33489] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 2 regionserver(s) complete 2023-07-20 16:54:34,167 INFO [Listener at localhost/33489] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 16:54:34,167 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:34,167 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:34,167 INFO [Listener at localhost/33489] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 16:54:34,168 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:34,168 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 16:54:34,168 INFO [Listener at localhost/33489] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 16:54:34,170 INFO [Listener at localhost/33489] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34355 2023-07-20 16:54:34,170 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:34,172 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:34,173 INFO [Listener at localhost/33489] zookeeper.RecoverableZooKeeper(93): Process identifier=master:34355 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:34,177 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:343550x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:34,178 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:34355-0x10183a7c131001d connected 2023-07-20 16:54:34,180 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 16:54:34,180 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:54:34,181 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 16:54:34,181 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34355 2023-07-20 16:54:34,181 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34355 2023-07-20 16:54:34,181 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34355 2023-07-20 16:54:34,182 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34355 2023-07-20 16:54:34,182 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34355 2023-07-20 16:54:34,182 INFO [Listener at localhost/33489] master.HMaster(444): hbase.rootdir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a, hbase.cluster.distributed=false 2023-07-20 16:54:34,183 DEBUG [pool-462-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-20 16:54:34,194 INFO [Listener at localhost/33489] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 16:54:34,195 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:34,195 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:34,195 INFO [Listener at localhost/33489] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 16:54:34,195 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:54:34,195 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 16:54:34,195 INFO [Listener at localhost/33489] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 16:54:34,197 INFO [Listener at localhost/33489] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46817 2023-07-20 16:54:34,197 INFO [Listener at localhost/33489] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 16:54:34,198 DEBUG [Listener at localhost/33489] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 16:54:34,199 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:34,200 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:34,201 INFO [Listener at localhost/33489] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46817 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:34,204 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:468170x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:34,205 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:468170x0, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 16:54:34,206 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46817-0x10183a7c131001e connected 2023-07-20 16:54:34,206 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:46817-0x10183a7c131001e, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:54:34,207 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:46817-0x10183a7c131001e, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 16:54:34,211 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46817 2023-07-20 16:54:34,211 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46817 2023-07-20 16:54:34,211 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46817 2023-07-20 16:54:34,211 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46817 2023-07-20 16:54:34,212 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46817 2023-07-20 16:54:34,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,34355,1689872074166 2023-07-20 16:54:34,217 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-20 16:54:34,217 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,34355,1689872074166 2023-07-20 16:54:34,219 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 16:54:34,219 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:46817-0x10183a7c131001e, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 16:54:34,219 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:54:34,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-20 16:54:34,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,34355,1689872074166 from backup master directory 2023-07-20 16:54:34,222 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-20 16:54:34,223 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,34355,1689872074166 2023-07-20 16:54:34,223 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-20 16:54:34,223 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 16:54:34,223 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,34355,1689872074166 2023-07-20 16:54:34,237 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:54:34,264 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x7a1d51e8 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:34,273 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@56a3cca8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:34,274 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 16:54:34,274 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-20 16:54:34,274 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:54:34,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:54:34,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,45867,1689872065106 to hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,45867,1689872065106-dead as it is dead 2023-07-20 16:54:34,283 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,45867,1689872065106-dead/jenkins-hbase4.apache.org%2C45867%2C1689872065106.jenkins-hbase4.apache.org%2C45867%2C1689872065106.regiongroup-0.1689872065391 2023-07-20 16:54:34,288 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,45867,1689872065106-dead/jenkins-hbase4.apache.org%2C45867%2C1689872065106.jenkins-hbase4.apache.org%2C45867%2C1689872065106.regiongroup-0.1689872065391 after 5ms 2023-07-20 16:54:34,289 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,45867,1689872065106-dead/jenkins-hbase4.apache.org%2C45867%2C1689872065106.jenkins-hbase4.apache.org%2C45867%2C1689872065106.regiongroup-0.1689872065391 to hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C45867%2C1689872065106.jenkins-hbase4.apache.org%2C45867%2C1689872065106.regiongroup-0.1689872065391 2023-07-20 16:54:34,289 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,45867,1689872065106-dead 2023-07-20 16:54:34,289 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,34355,1689872074166 2023-07-20 16:54:34,290 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:34,292 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34355%2C1689872074166.jenkins-hbase4.apache.org%2C34355%2C1689872074166.regiongroup-0, suffix=, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,34355,1689872074166, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/oldWALs, maxLogs=10 2023-07-20 16:54:34,302 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,34355,1689872074166/jenkins-hbase4.apache.org%2C34355%2C1689872074166.jenkins-hbase4.apache.org%2C34355%2C1689872074166.regiongroup-0.1689872074292 2023-07-20 16:54:34,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35885,DS-21386278-ed61-414b-9d04-87c474a58f9b,DISK], DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK], DatanodeInfoWithStorage[127.0.0.1:36101,DS-69ae4973-7589-41c7-bb37-0af0ca263f5e,DISK]] 2023-07-20 16:54:34,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:34,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:34,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:34,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:34,304 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:34,305 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-20 16:54:34,306 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-20 16:54:34,315 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/56183e65ade64a5ab9377e2784bcf14d 2023-07-20 16:54:34,316 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:34,317 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-20 16:54:34,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C45867%2C1689872065106.jenkins-hbase4.apache.org%2C45867%2C1689872065106.regiongroup-0.1689872065391 2023-07-20 16:54:34,322 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 131, firstSequenceIdInLog=3, maxSequenceIdInLog=118, path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C45867%2C1689872065106.jenkins-hbase4.apache.org%2C45867%2C1689872065106.regiongroup-0.1689872065391 2023-07-20 16:54:34,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C45867%2C1689872065106.jenkins-hbase4.apache.org%2C45867%2C1689872065106.regiongroup-0.1689872065391 2023-07-20 16:54:34,327 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:54:34,329 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=1 2023-07-20 16:54:34,330 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=119; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64721729, jitterRate=-0.035571083426475525}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 16:54:34,330 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 16:54:34,330 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-20 16:54:34,332 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-20 16:54:34,332 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-20 16:54:34,332 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-20 16:54:34,332 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-20 16:54:34,347 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-20 16:54:34,347 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-20 16:54:34,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-20 16:54:34,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-20 16:54:34,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-20 16:54:34,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-20 16:54:34,348 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 16 msec 2023-07-20 16:54:34,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-20 16:54:34,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-20 16:54:34,350 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,45493,1689872065148, table=hbase:meta, region=1588230740 2023-07-20 16:54:34,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 4 possibly 'live' servers, and 0 'splitting'. 2023-07-20 16:54:34,353 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,45493,1689872065148 already deleted, retry=false 2023-07-20 16:54:34,353 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,45493,1689872065148 on jenkins-hbase4.apache.org,34355,1689872074166 2023-07-20 16:54:34,362 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,45493,1689872065148, splitWal=true, meta=true 2023-07-20 16:54:34,362 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=15 for jenkins-hbase4.apache.org,45493,1689872065148 (carryingMeta=true) jenkins-hbase4.apache.org,45493,1689872065148/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@7e55d560[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-20 16:54:34,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,43901,1689872065207 already deleted, retry=false 2023-07-20 16:54:34,364 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,43901,1689872065207 on jenkins-hbase4.apache.org,34355,1689872074166 2023-07-20 16:54:34,366 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=16, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,43901,1689872065207, splitWal=true, meta=false 2023-07-20 16:54:34,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=16 for jenkins-hbase4.apache.org,43901,1689872065207 (carryingMeta=false) jenkins-hbase4.apache.org,43901,1689872065207/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@39de4fc6[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-20 16:54:34,371 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,45821,1689872065238 already deleted, retry=false 2023-07-20 16:54:34,371 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,45821,1689872065238 on jenkins-hbase4.apache.org,34355,1689872074166 2023-07-20 16:54:34,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,45821,1689872065238, splitWal=true, meta=false 2023-07-20 16:54:34,372 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=17 for jenkins-hbase4.apache.org,45821,1689872065238 (carryingMeta=false) jenkins-hbase4.apache.org,45821,1689872065238/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@69f06348[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-20 16:54:34,374 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46685,1689872065178 already deleted, retry=false 2023-07-20 16:54:34,374 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,46685,1689872065178 on jenkins-hbase4.apache.org,34355,1689872074166 2023-07-20 16:54:34,374 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,46685,1689872065178, splitWal=true, meta=false 2023-07-20 16:54:34,375 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,46685,1689872065178 (carryingMeta=false) jenkins-hbase4.apache.org,46685,1689872065178/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@3fcf6b21[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-20 16:54:34,385 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-20 16:54:34,385 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-20 16:54:34,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-20 16:54:34,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-20 16:54:34,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-20 16:54:34,387 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-20 16:54:34,387 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-20 16:54:34,388 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-20 16:54:34,389 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 16:54:34,390 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:46817-0x10183a7c131001e, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 16:54:34,390 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:54:34,390 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,34355,1689872074166, sessionid=0x10183a7c131001d, setting cluster-up flag (Was=false) 2023-07-20 16:54:34,394 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-20 16:54:34,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34355,1689872074166 2023-07-20 16:54:34,398 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-20 16:54:34,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34355,1689872074166 2023-07-20 16:54:34,400 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.hbase-snapshot/.tmp 2023-07-20 16:54:34,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:54:34,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:54:34,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:54:34,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:54:34,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-20 16:54:34,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:34,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 16:54:34,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:34,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689872104402 2023-07-20 16:54:34,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-20 16:54:34,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-20 16:54:34,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-20 16:54:34,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-20 16:54:34,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-20 16:54:34,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-20 16:54:34,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:34,404 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-20 16:54:34,405 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-20 16:54:34,405 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-20 16:54:34,406 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-20 16:54:34,407 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-20 16:54:34,408 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45493,1689872065148; numProcessing=1 2023-07-20 16:54:34,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872074407,5,FailOnTimeoutGroup] 2023-07-20 16:54:34,408 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43901,1689872065207; numProcessing=2 2023-07-20 16:54:34,408 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45821,1689872065238; numProcessing=3 2023-07-20 16:54:34,408 DEBUG [PEWorker-4] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46685,1689872065178; numProcessing=4 2023-07-20 16:54:34,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872074408,5,FailOnTimeoutGroup] 2023-07-20 16:54:34,408 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:34,408 INFO [PEWorker-4] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46685,1689872065178, splitWal=true, meta=false 2023-07-20 16:54:34,408 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-20 16:54:34,408 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=16, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43901,1689872065207, splitWal=true, meta=false 2023-07-20 16:54:34,408 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=17, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45821,1689872065238, splitWal=true, meta=false 2023-07-20 16:54:34,408 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=15, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45493,1689872065148, splitWal=true, meta=true 2023-07-20 16:54:34,408 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:34,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:34,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689872074409, completionTime=-1 2023-07-20 16:54:34,409 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-20 16:54:34,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-20 16:54:34,410 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45493,1689872065148, splitWal=true, meta=true, isMeta: true 2023-07-20 16:54:34,412 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148-splitting 2023-07-20 16:54:34,413 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148-splitting dir is empty, no logs to split. 2023-07-20 16:54:34,413 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,45493,1689872065148 WAL count=0, meta=true 2023-07-20 16:54:34,415 INFO [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(951): ClusterId : de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:54:34,415 DEBUG [RS:0;jenkins-hbase4:46817] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 16:54:34,417 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148-splitting dir is empty, no logs to split. 2023-07-20 16:54:34,417 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,45493,1689872065148 WAL count=0, meta=true 2023-07-20 16:54:34,417 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,45493,1689872065148 WAL splitting is done? wals=0, meta=true 2023-07-20 16:54:34,418 DEBUG [RS:0;jenkins-hbase4:46817] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 16:54:34,418 DEBUG [RS:0;jenkins-hbase4:46817] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 16:54:34,419 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-20 16:54:34,420 DEBUG [RS:0;jenkins-hbase4:46817] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 16:54:34,421 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-20 16:54:34,422 DEBUG [RS:0;jenkins-hbase4:46817] zookeeper.ReadOnlyZKClient(139): Connect 0x1c0460bd to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:34,423 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=19, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 16:54:34,434 DEBUG [RS:0;jenkins-hbase4:46817] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@12c39d4d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:34,434 DEBUG [RS:0;jenkins-hbase4:46817] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@16fc0bcd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 16:54:34,442 DEBUG [RS:0;jenkins-hbase4:46817] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:46817 2023-07-20 16:54:34,443 INFO [RS:0;jenkins-hbase4:46817] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 16:54:34,443 INFO [RS:0;jenkins-hbase4:46817] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 16:54:34,443 DEBUG [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 16:54:34,443 INFO [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,34355,1689872074166 with isa=jenkins-hbase4.apache.org/172.31.14.131:46817, startcode=1689872074194 2023-07-20 16:54:34,443 DEBUG [RS:0;jenkins-hbase4:46817] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 16:54:34,449 INFO [RS-EventLoopGroup-11-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48177, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 16:54:34,451 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34355] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:54:34,451 DEBUG [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a 2023-07-20 16:54:34,451 DEBUG [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33767 2023-07-20 16:54:34,451 DEBUG [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 16:54:34,454 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:54:34,455 DEBUG [RS:0;jenkins-hbase4:46817] zookeeper.ZKUtil(162): regionserver:46817-0x10183a7c131001e, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:54:34,455 WARN [RS:0;jenkins-hbase4:46817] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 16:54:34,455 INFO [RS:0;jenkins-hbase4:46817] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:54:34,455 INFO [RS:0;jenkins-hbase4:46817] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:54:34,455 DEBUG [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:54:34,456 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46817,1689872074194] 2023-07-20 16:54:34,459 DEBUG [RS:0;jenkins-hbase4:46817] zookeeper.ZKUtil(162): regionserver:46817-0x10183a7c131001e, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:54:34,459 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-20 16:54:34,460 DEBUG [RS:0;jenkins-hbase4:46817] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 16:54:34,460 INFO [RS:0;jenkins-hbase4:46817] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 16:54:34,462 INFO [RS:0;jenkins-hbase4:46817] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 16:54:34,462 INFO [RS:0;jenkins-hbase4:46817] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 16:54:34,463 INFO [RS:0;jenkins-hbase4:46817] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:34,463 INFO [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 16:54:34,464 INFO [RS:0;jenkins-hbase4:46817] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:34,464 DEBUG [RS:0;jenkins-hbase4:46817] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:34,464 DEBUG [RS:0;jenkins-hbase4:46817] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:34,464 DEBUG [RS:0;jenkins-hbase4:46817] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:34,464 DEBUG [RS:0;jenkins-hbase4:46817] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:34,464 DEBUG [RS:0;jenkins-hbase4:46817] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:34,464 DEBUG [RS:0;jenkins-hbase4:46817] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 16:54:34,464 DEBUG [RS:0;jenkins-hbase4:46817] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:34,464 DEBUG [RS:0;jenkins-hbase4:46817] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:34,465 DEBUG [RS:0;jenkins-hbase4:46817] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:34,465 DEBUG [RS:0;jenkins-hbase4:46817] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:54:34,465 INFO [RS:0;jenkins-hbase4:46817] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:34,465 INFO [RS:0;jenkins-hbase4:46817] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:34,466 INFO [RS:0;jenkins-hbase4:46817] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:34,477 INFO [RS:0;jenkins-hbase4:46817] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 16:54:34,477 INFO [RS:0;jenkins-hbase4:46817] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46817,1689872074194-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:34,489 INFO [RS:0;jenkins-hbase4:46817] regionserver.Replication(203): jenkins-hbase4.apache.org,46817,1689872074194 started 2023-07-20 16:54:34,489 INFO [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46817,1689872074194, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46817, sessionid=0x10183a7c131001e 2023-07-20 16:54:34,489 DEBUG [RS:0;jenkins-hbase4:46817] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 16:54:34,489 DEBUG [RS:0;jenkins-hbase4:46817] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:54:34,489 DEBUG [RS:0;jenkins-hbase4:46817] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46817,1689872074194' 2023-07-20 16:54:34,489 DEBUG [RS:0;jenkins-hbase4:46817] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-20 16:54:34,489 DEBUG [RS:0;jenkins-hbase4:46817] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-20 16:54:34,490 DEBUG [RS:0;jenkins-hbase4:46817] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 16:54:34,490 DEBUG [RS:0;jenkins-hbase4:46817] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 16:54:34,490 DEBUG [RS:0;jenkins-hbase4:46817] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:54:34,490 DEBUG [RS:0;jenkins-hbase4:46817] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46817,1689872074194' 2023-07-20 16:54:34,490 DEBUG [RS:0;jenkins-hbase4:46817] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-20 16:54:34,490 DEBUG [RS:0;jenkins-hbase4:46817] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-20 16:54:34,491 DEBUG [RS:0;jenkins-hbase4:46817] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 16:54:34,491 INFO [RS:0;jenkins-hbase4:46817] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 16:54:34,491 INFO [RS:0;jenkins-hbase4:46817] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 16:54:34,577 DEBUG [jenkins-hbase4:34355] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-20 16:54:34,578 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46817,1689872074194, state=OPENING 2023-07-20 16:54:34,580 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-20 16:54:34,580 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-20 16:54:34,580 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46817,1689872074194}] 2023-07-20 16:54:34,591 INFO [RS:0;jenkins-hbase4:46817] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:34,593 INFO [RS:0;jenkins-hbase4:46817] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46817%2C1689872074194.jenkins-hbase4.apache.org%2C46817%2C1689872074194.regiongroup-0, suffix=, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs, maxLogs=10 2023-07-20 16:54:34,610 INFO [RS:0;jenkins-hbase4:46817] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194/jenkins-hbase4.apache.org%2C46817%2C1689872074194.jenkins-hbase4.apache.org%2C46817%2C1689872074194.regiongroup-0.1689872074594 2023-07-20 16:54:34,610 DEBUG [RS:0;jenkins-hbase4:46817] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK], DatanodeInfoWithStorage[127.0.0.1:36101,DS-69ae4973-7589-41c7-bb37-0af0ca263f5e,DISK], DatanodeInfoWithStorage[127.0.0.1:35885,DS-21386278-ed61-414b-9d04-87c474a58f9b,DISK]] 2023-07-20 16:54:34,735 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:54:34,735 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 16:54:34,738 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45252, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 16:54:34,742 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-20 16:54:34,742 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:54:34,742 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:54:34,742 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:34,744 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46817%2C1689872074194.meta, suffix=.meta, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs, maxLogs=10 2023-07-20 16:54:34,754 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194/jenkins-hbase4.apache.org%2C46817%2C1689872074194.meta.1689872074744.meta 2023-07-20 16:54:34,754 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35847,DS-e3134465-ba87-4999-ba03-bbcb207f3c3b,DISK], DatanodeInfoWithStorage[127.0.0.1:35885,DS-21386278-ed61-414b-9d04-87c474a58f9b,DISK], DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK]] 2023-07-20 16:54:34,754 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:34,754 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-20 16:54:34,754 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-20 16:54:34,755 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-20 16:54:34,755 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-20 16:54:34,756 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:34,756 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-20 16:54:34,756 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-20 16:54:34,757 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-20 16:54:34,758 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info 2023-07-20 16:54:34,758 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info 2023-07-20 16:54:34,759 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-20 16:54:34,766 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info/730b9259ce88426992f0c3d8aad85399 2023-07-20 16:54:34,766 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:34,766 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-20 16:54:34,767 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/rep_barrier 2023-07-20 16:54:34,767 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/rep_barrier 2023-07-20 16:54:34,768 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-20 16:54:34,768 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:34,768 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-20 16:54:34,769 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/table 2023-07-20 16:54:34,769 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/table 2023-07-20 16:54:34,770 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-20 16:54:34,776 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/table/1619f7b97a8e4850aae967c3b3d7b384 2023-07-20 16:54:34,776 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:34,777 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740 2023-07-20 16:54:34,778 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740 2023-07-20 16:54:34,781 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-20 16:54:34,782 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-20 16:54:34,783 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67804276, jitterRate=0.010362446308135986}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-20 16:54:34,783 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-20 16:54:34,790 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=20, masterSystemTime=1689872074735 2023-07-20 16:54:34,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-20 16:54:34,796 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-20 16:54:34,796 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46817,1689872074194, state=OPEN 2023-07-20 16:54:34,798 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-20 16:54:34,799 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-20 16:54:34,801 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-20 16:54:34,801 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46817,1689872074194 in 219 msec 2023-07-20 16:54:34,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=15 2023-07-20 16:54:34,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 383 msec 2023-07-20 16:54:35,962 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1553ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-20 16:54:37,464 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3055ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-20 16:54:38,916 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4507ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-20 16:54:38,917 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-20 16:54:38,922 DEBUG [hconnection-0x649af689-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:38,924 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45256, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:54:38,934 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=7ebeef24b783c4189ed905a5659b7e20, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,45821,1689872065238, regionLocation=jenkins-hbase4.apache.org,45821,1689872065238, openSeqNum=2 2023-07-20 16:54:38,934 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=058a0bfd9d8bf39d8c91089a151cb277, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,45493,1689872065148, regionLocation=jenkins-hbase4.apache.org,45493,1689872065148, openSeqNum=2 2023-07-20 16:54:38,934 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=c96c52d963776c1249f8d28dc1cee95e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43901,1689872065207, regionLocation=jenkins-hbase4.apache.org,43901,1689872065207, openSeqNum=2 2023-07-20 16:54:38,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-20 16:54:38,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689872138934 2023-07-20 16:54:38,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689872198934 2023-07-20 16:54:38,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 17 msec 2023-07-20 16:54:38,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34355,1689872074166-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:38,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34355,1689872074166-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:38,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34355,1689872074166-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:38,964 INFO [PEWorker-1] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,46685,1689872065178 had 0 regions 2023-07-20 16:54:38,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:34355, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:38,964 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,43901,1689872065207 had 1 regions 2023-07-20 16:54:38,965 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-20 16:54:38,964 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,45493,1689872065148 had 2 regions 2023-07-20 16:54:38,964 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,45821,1689872065238 had 1 regions 2023-07-20 16:54:38,966 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. is NOT online; state={7ebeef24b783c4189ed905a5659b7e20 state=OPEN, ts=1689872078934, server=jenkins-hbase4.apache.org,45821,1689872065238}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-20 16:54:38,967 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45493,1689872065148, splitWal=true, meta=true, isMeta: false 2023-07-20 16:54:38,967 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46685,1689872065178, splitWal=true, meta=false, isMeta: false 2023-07-20 16:54:38,967 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=16, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,43901,1689872065207, splitWal=true, meta=false, isMeta: false 2023-07-20 16:54:38,967 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=17, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45821,1689872065238, splitWal=true, meta=false, isMeta: false 2023-07-20 16:54:38,970 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46685,1689872065178-splitting 2023-07-20 16:54:38,970 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148-splitting dir is empty, no logs to split. 2023-07-20 16:54:38,971 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,45493,1689872065148 WAL count=0, meta=false 2023-07-20 16:54:38,972 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,45821,1689872065238/hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20., unknown_server=jenkins-hbase4.apache.org,45493,1689872065148/t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277., unknown_server=jenkins-hbase4.apache.org,43901,1689872065207/t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:38,973 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46685,1689872065178-splitting dir is empty, no logs to split. 2023-07-20 16:54:38,973 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46685,1689872065178 WAL count=0, meta=false 2023-07-20 16:54:38,973 DEBUG [PEWorker-3] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,43901,1689872065207-splitting 2023-07-20 16:54:38,974 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,43901,1689872065207-splitting dir is empty, no logs to split. 2023-07-20 16:54:38,974 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,43901,1689872065207 WAL count=0, meta=false 2023-07-20 16:54:38,974 DEBUG [PEWorker-2] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45821,1689872065238-splitting 2023-07-20 16:54:38,975 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45821,1689872065238-splitting dir is empty, no logs to split. 2023-07-20 16:54:38,975 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,45821,1689872065238 WAL count=0, meta=false 2023-07-20 16:54:38,976 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45493,1689872065148-splitting dir is empty, no logs to split. 2023-07-20 16:54:38,976 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,45493,1689872065148 WAL count=0, meta=false 2023-07-20 16:54:38,976 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,45493,1689872065148 WAL splitting is done? wals=0, meta=false 2023-07-20 16:54:38,977 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46685,1689872065178-splitting dir is empty, no logs to split. 2023-07-20 16:54:38,977 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46685,1689872065178 WAL count=0, meta=false 2023-07-20 16:54:38,977 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46685,1689872065178 WAL splitting is done? wals=0, meta=false 2023-07-20 16:54:38,978 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,43901,1689872065207-splitting dir is empty, no logs to split. 2023-07-20 16:54:38,978 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,43901,1689872065207 WAL count=0, meta=false 2023-07-20 16:54:38,978 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,43901,1689872065207 WAL splitting is done? wals=0, meta=false 2023-07-20 16:54:38,980 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45821,1689872065238-splitting dir is empty, no logs to split. 2023-07-20 16:54:38,980 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,45821,1689872065238 WAL count=0, meta=false 2023-07-20 16:54:38,980 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,45821,1689872065238 WAL splitting is done? wals=0, meta=false 2023-07-20 16:54:38,981 INFO [PEWorker-1] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,46685,1689872065178 failed, ignore...File hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46685,1689872065178-splitting does not exist. 2023-07-20 16:54:38,982 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=058a0bfd9d8bf39d8c91089a151cb277, ASSIGN}] 2023-07-20 16:54:38,983 INFO [PEWorker-3] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,43901,1689872065207 failed, ignore...File hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,43901,1689872065207-splitting does not exist. 2023-07-20 16:54:38,983 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,46685,1689872065178 after splitting done 2023-07-20 16:54:38,983 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,46685,1689872065178 from processing; numProcessing=3 2023-07-20 16:54:38,984 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=058a0bfd9d8bf39d8c91089a151cb277, ASSIGN 2023-07-20 16:54:38,984 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=c96c52d963776c1249f8d28dc1cee95e, ASSIGN}] 2023-07-20 16:54:38,984 INFO [PEWorker-2] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,45821,1689872065238 failed, ignore...File hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,45821,1689872065238-splitting does not exist. 2023-07-20 16:54:38,985 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=21, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=058a0bfd9d8bf39d8c91089a151cb277, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 16:54:38,986 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=7ebeef24b783c4189ed905a5659b7e20, ASSIGN}] 2023-07-20 16:54:38,986 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=c96c52d963776c1249f8d28dc1cee95e, ASSIGN 2023-07-20 16:54:38,986 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,46685,1689872065178, splitWal=true, meta=false in 4.6090 sec 2023-07-20 16:54:38,988 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=7ebeef24b783c4189ed905a5659b7e20, ASSIGN 2023-07-20 16:54:38,988 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=16, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=c96c52d963776c1249f8d28dc1cee95e, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 16:54:38,989 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=7ebeef24b783c4189ed905a5659b7e20, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 16:54:38,989 DEBUG [jenkins-hbase4:34355] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-20 16:54:38,992 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=c96c52d963776c1249f8d28dc1cee95e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:54:38,992 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=058a0bfd9d8bf39d8c91089a151cb277, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:54:38,993 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872078992"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872078992"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872078992"}]},"ts":"1689872078992"} 2023-07-20 16:54:38,993 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872078992"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872078992"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872078992"}]},"ts":"1689872078992"} 2023-07-20 16:54:38,993 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=7ebeef24b783c4189ed905a5659b7e20, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:54:38,993 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689872078993"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872078993"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872078993"}]},"ts":"1689872078993"} 2023-07-20 16:54:38,996 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=22, state=RUNNABLE; OpenRegionProcedure c96c52d963776c1249f8d28dc1cee95e, server=jenkins-hbase4.apache.org,46817,1689872074194}] 2023-07-20 16:54:38,997 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=21, state=RUNNABLE; OpenRegionProcedure 058a0bfd9d8bf39d8c91089a151cb277, server=jenkins-hbase4.apache.org,46817,1689872074194}] 2023-07-20 16:54:38,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=23, state=RUNNABLE; OpenRegionProcedure 7ebeef24b783c4189ed905a5659b7e20, server=jenkins-hbase4.apache.org,46817,1689872074194}] 2023-07-20 16:54:39,065 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 16:54:39,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:39,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:54:39,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46817%2C1689872074194.jenkins-hbase4.apache.org%2C46817%2C1689872074194.regiongroup-1, suffix=, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs, maxLogs=10 2023-07-20 16:54:39,163 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:39,163 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7ebeef24b783c4189ed905a5659b7e20, NAME => 'hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:39,163 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:39,164 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:39,164 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:39,164 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:39,165 INFO [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:39,167 DEBUG [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/info 2023-07-20 16:54:39,167 DEBUG [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/info 2023-07-20 16:54:39,168 INFO [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7ebeef24b783c4189ed905a5659b7e20 columnFamilyName info 2023-07-20 16:54:39,174 DEBUG [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/info/ab70d834ddd448fe9bb282fb38cf6ad4 2023-07-20 16:54:39,175 INFO [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] regionserver.HStore(310): Store=7ebeef24b783c4189ed905a5659b7e20/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:39,175 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:39,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194/jenkins-hbase4.apache.org%2C46817%2C1689872074194.jenkins-hbase4.apache.org%2C46817%2C1689872074194.regiongroup-1.1689872079157 2023-07-20 16:54:39,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35885,DS-21386278-ed61-414b-9d04-87c474a58f9b,DISK], DatanodeInfoWithStorage[127.0.0.1:36101,DS-69ae4973-7589-41c7-bb37-0af0ca263f5e,DISK], DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK]] 2023-07-20 16:54:39,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c96c52d963776c1249f8d28dc1cee95e, NAME => 't2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:39,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:39,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:39,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:39,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:39,178 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:39,179 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:39,180 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1 2023-07-20 16:54:39,180 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1 2023-07-20 16:54:39,181 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c96c52d963776c1249f8d28dc1cee95e columnFamilyName cf1 2023-07-20 16:54:39,182 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:54:39,183 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7ebeef24b783c4189ed905a5659b7e20; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68196614, jitterRate=0.016208738088607788}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 16:54:39,183 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7ebeef24b783c4189ed905a5659b7e20: 2023-07-20 16:54:39,186 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20., pid=26, masterSystemTime=1689872079148 2023-07-20 16:54:39,189 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:39,189 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:54:39,190 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=7ebeef24b783c4189ed905a5659b7e20, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:54:39,190 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689872079190"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872079190"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872079190"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872079190"}]},"ts":"1689872079190"} 2023-07-20 16:54:39,193 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1/6419e14a3bbe44399930ef7d7a7c0e8b 2023-07-20 16:54:39,193 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(310): Store=c96c52d963776c1249f8d28dc1cee95e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:39,194 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:39,195 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=23 2023-07-20 16:54:39,195 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=23, state=SUCCESS; OpenRegionProcedure 7ebeef24b783c4189ed905a5659b7e20, server=jenkins-hbase4.apache.org,46817,1689872074194 in 193 msec 2023-07-20 16:54:39,195 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/norep 2023-07-20 16:54:39,195 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/norep 2023-07-20 16:54:39,195 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c96c52d963776c1249f8d28dc1cee95e columnFamilyName norep 2023-07-20 16:54:39,196 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(310): Store=c96c52d963776c1249f8d28dc1cee95e/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:39,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:39,197 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=17 2023-07-20 16:54:39,197 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,45821,1689872065238 after splitting done 2023-07-20 16:54:39,197 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=7ebeef24b783c4189ed905a5659b7e20, ASSIGN in 210 msec 2023-07-20 16:54:39,198 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,45821,1689872065238 from processing; numProcessing=2 2023-07-20 16:54:39,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:39,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,45821,1689872065238, splitWal=true, meta=false in 4.8270 sec 2023-07-20 16:54:39,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 16:54:39,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:54:39,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c96c52d963776c1249f8d28dc1cee95e; next sequenceid=12; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72414543, jitterRate=0.07906077802181244}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 16:54:39,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c96c52d963776c1249f8d28dc1cee95e: 2023-07-20 16:54:39,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e., pid=24, masterSystemTime=1689872079148 2023-07-20 16:54:39,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:39,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:54:39,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:39,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 058a0bfd9d8bf39d8c91089a151cb277, NAME => 't1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:54:39,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:39,208 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=c96c52d963776c1249f8d28dc1cee95e, regionState=OPEN, openSeqNum=12, regionLocation=jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:54:39,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:54:39,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:39,209 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872079208"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872079208"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872079208"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872079208"}]},"ts":"1689872079208"} 2023-07-20 16:54:39,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:39,213 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=22 2023-07-20 16:54:39,213 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=22, state=SUCCESS; OpenRegionProcedure c96c52d963776c1249f8d28dc1cee95e, server=jenkins-hbase4.apache.org,46817,1689872074194 in 214 msec 2023-07-20 16:54:39,213 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:39,215 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1 2023-07-20 16:54:39,215 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1 2023-07-20 16:54:39,215 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 058a0bfd9d8bf39d8c91089a151cb277 columnFamilyName cf1 2023-07-20 16:54:39,215 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=16 2023-07-20 16:54:39,216 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,43901,1689872065207 after splitting done 2023-07-20 16:54:39,216 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=16, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=c96c52d963776c1249f8d28dc1cee95e, ASSIGN in 229 msec 2023-07-20 16:54:39,216 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,43901,1689872065207 from processing; numProcessing=1 2023-07-20 16:54:39,218 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=16, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,43901,1689872065207, splitWal=true, meta=false in 4.8510 sec 2023-07-20 16:54:39,223 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1/806acc8751334bdeaac05f420a6fcdb3 2023-07-20 16:54:39,223 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(310): Store=058a0bfd9d8bf39d8c91089a151cb277/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:39,223 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:39,224 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/norep 2023-07-20 16:54:39,224 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/norep 2023-07-20 16:54:39,225 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 058a0bfd9d8bf39d8c91089a151cb277 columnFamilyName norep 2023-07-20 16:54:39,225 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(310): Store=058a0bfd9d8bf39d8c91089a151cb277/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:54:39,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:39,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:39,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 16:54:39,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:54:39,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 058a0bfd9d8bf39d8c91089a151cb277; next sequenceid=11; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71563815, jitterRate=0.06638394296169281}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 16:54:39,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 058a0bfd9d8bf39d8c91089a151cb277: 2023-07-20 16:54:39,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277., pid=25, masterSystemTime=1689872079148 2023-07-20 16:54:39,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:39,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:54:39,237 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=058a0bfd9d8bf39d8c91089a151cb277, regionState=OPEN, openSeqNum=11, regionLocation=jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:54:39,237 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872079237"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872079237"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872079237"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872079237"}]},"ts":"1689872079237"} 2023-07-20 16:54:39,243 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=21 2023-07-20 16:54:39,243 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=21, state=SUCCESS; OpenRegionProcedure 058a0bfd9d8bf39d8c91089a151cb277, server=jenkins-hbase4.apache.org,46817,1689872074194 in 242 msec 2023-07-20 16:54:39,246 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=15 2023-07-20 16:54:39,246 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,45493,1689872065148 after splitting done 2023-07-20 16:54:39,246 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=058a0bfd9d8bf39d8c91089a151cb277, ASSIGN in 261 msec 2023-07-20 16:54:39,246 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,45493,1689872065148 from processing; numProcessing=0 2023-07-20 16:54:39,249 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,45493,1689872065148, splitWal=true, meta=true in 4.8920 sec 2023-07-20 16:54:39,489 INFO [jenkins-hbase4:46817Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:54:39,968 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-20 16:54:39,982 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-20 16:54:39,984 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-20 16:54:39,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.761sec 2023-07-20 16:54:39,985 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-20 16:54:39,985 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-20 16:54:39,985 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-20 16:54:39,985 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34355,1689872074166-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-20 16:54:39,985 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34355,1689872074166-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-20 16:54:39,988 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-20 16:54:40,019 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(139): Connect 0x10e8b0f3 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:40,025 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3d034449, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:40,027 DEBUG [hconnection-0xd00b149-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:40,029 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45272, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:54:40,032 INFO [Listener at localhost/33489] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-20 16:54:40,032 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x10e8b0f3 to 127.0.0.1:56032 2023-07-20 16:54:40,033 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:54:40,033 INFO [Listener at localhost/33489] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:34355 after: jenkins-hbase4.apache.org:34355 2023-07-20 16:54:40,034 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(139): Connect 0x58813696 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:40,039 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@50834e36, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:40,461 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-20 16:54:40,461 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-20 16:54:40,462 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-20 16:54:40,462 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-20 16:54:40,541 DEBUG [hconnection-0x104b72a7-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:40,545 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55082, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:54:40,570 INFO [Listener at localhost/33489] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689872080570 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:40,574 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): syncupReplication16898720805700x0, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:40,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689872080570-0x10183a7c1310023 connected Start Replication Server start 2023-07-20 16:54:40,578 INFO [Listener at localhost/33489] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:54:40,578 INFO [Listener at localhost/33489] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:54:40,590 INFO [Listener at localhost/33489] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-20 16:54:40,591 DEBUG [Listener at localhost/33489] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 16:54:40,602 INFO [Listener at localhost/33489] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-20 16:54:40,603 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,33103,1689872061305, jenkins-hbase4.apache.org,34183,1689872061357] other RSs: [] 2023-07-20 16:54:40,623 DEBUG [Listener at localhost/33489.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x7f4fbb9c to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:40,633 DEBUG [Listener at localhost/33489.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@1183c429, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:40,633 INFO [Listener at localhost/33489.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:40,636 DEBUG [Listener at localhost/33489.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:40,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10183a7c1310025 connected 2023-07-20 16:54:40,637 INFO [Listener at localhost/33489.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=258804e0-584b-4fe6-b46a-945d0467644e to cluster=de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:54:44,466 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,33103,1689872061305/1's WALs to 1689872080578.syncuptool.replication.org,1234,1 2023-07-20 16:54:44,468 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0.1689872062821 with data PBUF\x08\xF8\xA9\x01 2023-07-20 16:54:44,468 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1.1689872067118 with data PBUF\x08\xDFU 2023-07-20 16:54:44,477 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,33103,1689872061305/1's WALs to 1689872080578.syncuptool.replication.org,1234,1 2023-07-20 16:54:44,489 INFO [jenkins-hbase4:46817Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:54:44,667 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 16:54:46,479 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,33103,1689872061305] 2023-07-20 16:54:46,480 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,33103,1689872061305] 2023-07-20 16:54:46,495 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,33103,1689872061305] 2023-07-20 16:54:46,495 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,33103,1689872061305, ReplicationSource: 1, currentBandwidth=0 2023-07-20 16:54:46,495 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:46,515 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305] zookeeper.ReadOnlyZKClient(139): Connect 0x2a9add19 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:46,522 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@41f9cf4f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:46,523 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:46,527 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:46,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10183a7c1310027 connected 2023-07-20 16:54:46,528 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,33103,1689872061305 (queues=2) is replicating from cluster=258804e0-584b-4fe6-b46a-945d0467644e to cluster=de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:54:46,528 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1 2023-07-20 16:54:46,532 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,33103,1689872061305, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 16:54:46,532 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0 2023-07-20 16:54:46,532 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1 2023-07-20 16:54:46,534 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,33103,1689872061305, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 16:54:46,534 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0 2023-07-20 16:54:46,540 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1.1689872067118 2023-07-20 16:54:46,540 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-20 16:54:46,540 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-20 16:54:46,541 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,33103,1689872061305 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/ea29850f58e32db8bff448bce6d2160a/105=[#edits: 50 = ],4120}], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1.1689872067118, lastWalPosition=15173, nbRowKeys=50, nbHFiles=0, heapSize=4120, lastSeqIds={}, endOfFile=true,usedBufferSize=4120] 2023-07-20 16:54:46,543 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0.1689872062821 2023-07-20 16:54:46,543 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceWALReader(162): Read 1 WAL entries eligible for replication 2023-07-20 16:54:46,543 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-20 16:54:46,544 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 16:54:46,544 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,33103,1689872061305 got entry batch from reader: WALEntryBatch [walEntries=[{t2_syncup/2627e10c9cf6db0694d7b55799cf6705/205=[#edits: 100 = ],8120}], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs/jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0.1689872062821, lastWalPosition=29258, nbRowKeys=100, nbHFiles=0, heapSize=8120, lastSeqIds={}, endOfFile=true,usedBufferSize=8120] 2023-07-20 16:54:46,550 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55090, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 16:54:46,552 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46817] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 16:54:46,552 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46817] zookeeper.ReadOnlyZKClient(139): Connect 0x5aa9a029 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:46,553 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=46817] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 16:54:46,558 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46817] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@546e450c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:46,561 DEBUG [hconnection-0x5051cb59-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:54:46,566 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55106, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=ClientService 2023-07-20 16:54:46,576 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46817] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 16:54:46,577 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1-jenkins-hbase4.apache.org,33103,1689872061305] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,33103,1689872061305] 2023-07-20 16:54:46,579 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1.1689872067118] 2023-07-20 16:54:46,580 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,33103,1689872061305 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 16:54:46,580 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=46817] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 16:54:46,580 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:46,581 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1-jenkins-hbase4.apache.org,33103,1689872061305] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,33103,1689872061305] 2023-07-20 16:54:46,582 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0.1689872062821] 2023-07-20 16:54:46,584 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,33103,1689872061305 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 16:54:46,585 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:46,681 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:46,684 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689872080570-0x10183a7c1310023, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689872080578.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:46,684 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-1,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,33103,1689872061305 with the following stats: Total replicated edits: 2, current progress: 2023-07-20 16:54:46,784 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,33103,1689872061305 2023-07-20 16:54:46,785 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1-jenkins-hbase4.apache.org,33103,1689872061305] zookeeper.ZKUtil(243): syncupReplication1689872080570-0x10183a7c1310023, quorum=127.0.0.1:56032, baseZNode=/1 Unable to list children of znode /1/replication/rs/1689872080578.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,33103,1689872061305 because node does not exist (not an error) 2023-07-20 16:54:46,785 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305.replicationSource.shipperjenkins-hbase4.apache.org%2C33103%2C1689872061305.jenkins-hbase4.apache.org%2C33103%2C1689872061305.regiongroup-0,1-jenkins-hbase4.apache.org,33103,1689872061305] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,33103,1689872061305 with the following stats: Total replicated edits: 2, current progress: 2023-07-20 16:54:49,489 INFO [jenkins-hbase4:46817Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:54:49,489 INFO [jenkins-hbase4:46817Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-20 16:54:49,813 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,34183,1689872061357/1's WALs to 1689872080578.syncuptool.replication.org,1234,1 2023-07-20 16:54:49,815 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1.1689872063702 with data PBUF\x08\x96\x0B 2023-07-20 16:54:49,816 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,34183,1689872061357/1's WALs to 1689872080578.syncuptool.replication.org,1234,1 2023-07-20 16:54:51,818 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,34183,1689872061357] 2023-07-20 16:54:51,818 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,34183,1689872061357] 2023-07-20 16:54:51,829 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,34183,1689872061357] 2023-07-20 16:54:51,829 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,34183,1689872061357, ReplicationSource: 1, currentBandwidth=0 2023-07-20 16:54:51,829 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:51,850 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357] zookeeper.ReadOnlyZKClient(139): Connect 0x23b44123 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:54:51,854 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@4d8ec66a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:54:51,855 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:54:51,856 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:54:51,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10183a7c131002a connected 2023-07-20 16:54:51,858 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,34183,1689872061357 (queues=1) is replicating from cluster=258804e0-584b-4fe6-b46a-945d0467644e to cluster=de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:54:51,858 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1 2023-07-20 16:54:51,859 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,34183,1689872061357, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 16:54:51,863 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357.replicationSource.shipperjenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1-jenkins-hbase4.apache.org,34183,1689872061357] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1 2023-07-20 16:54:51,869 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1-jenkins-hbase4.apache.org,34183,1689872061357] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs/jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1.1689872063702 2023-07-20 16:54:51,869 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1-jenkins-hbase4.apache.org,34183,1689872061357] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-20 16:54:51,869 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357.replicationSource.shipperjenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1-jenkins-hbase4.apache.org,34183,1689872061357] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,34183,1689872061357 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 16:54:51,870 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357.replicationSource.shipperjenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1-jenkins-hbase4.apache.org,34183,1689872061357] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:51,970 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357.replicationSource.shipperjenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1-jenkins-hbase4.apache.org,34183,1689872061357] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:51,971 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357.replicationSource.shipperjenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1-jenkins-hbase4.apache.org,34183,1689872061357] zookeeper.ZKUtil(162): syncupReplication1689872080570-0x10183a7c1310023, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/replication/rs/1689872080578.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,34183,1689872061357/jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1.1689872063702 2023-07-20 16:54:51,974 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689872080570-0x10183a7c1310023, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689872080578.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,34183,1689872061357/jenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1.1689872063702 2023-07-20 16:54:51,974 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689872080570-0x10183a7c1310023, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/replication/rs/1689872080578.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:51,974 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357.replicationSource.shipperjenkins-hbase4.apache.org%2C34183%2C1689872061357.jenkins-hbase4.apache.org%2C34183%2C1689872061357.regiongroup-1,1-jenkins-hbase4.apache.org,34183,1689872061357] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,34183,1689872061357 with the following stats: Total replicated edits: 0, current progress: 2023-07-20 16:54:51,974 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689872080570-0x10183a7c1310023, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689872080578.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,34183,1689872061357 2023-07-20 16:54:52,185 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 16:54:54,489 INFO [jenkins-hbase4:46817Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:54:54,489 INFO [jenkins-hbase4:46817Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-20 16:54:57,473 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 16:54:59,489 INFO [jenkins-hbase4:46817Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:54:59,489 INFO [jenkins-hbase4:46817Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-20 16:55:00,605 INFO [Listener at localhost/33489] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-20 16:55:00,709 DEBUG [Listener at localhost/33489.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310025, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:00,709 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7f4fbb9c to 127.0.0.1:56032 2023-07-20 16:55:00,710 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:00,912 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689872080570-0x10183a7c1310023, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:00,912 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689872080570-0x10183a7c1310023, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:00,928 INFO [Listener at localhost/33489] replication.TestReplicationSyncUpTool(203): SyncUpAfterDelete succeeded at retry = 0 2023-07-20 16:55:00,928 DEBUG [Listener at localhost/33489] replication.TestReplicationSyncUpTool(215): mimicSyncUpAfterPut 2023-07-20 16:55:00,930 INFO [Listener at localhost/33489] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-20 16:55:00,930 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:55:00,930 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 16:55:00,930 INFO [Listener at localhost/33489] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 16:55:00,931 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:55:00,931 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 16:55:00,931 INFO [Listener at localhost/33489] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 16:55:00,933 INFO [Listener at localhost/33489] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37313 2023-07-20 16:55:00,933 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:55:00,934 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:55:00,935 INFO [Listener at localhost/33489] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37313 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:55:00,939 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:373130x0, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:55:00,940 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:37313-0x10183a7c131002b connected 2023-07-20 16:55:00,942 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-20 16:55:00,943 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 16:55:00,943 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-20 16:55:00,949 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37313 2023-07-20 16:55:00,949 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37313 2023-07-20 16:55:00,950 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37313 2023-07-20 16:55:00,951 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37313 2023-07-20 16:55:00,954 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37313 2023-07-20 16:55:00,954 INFO [Listener at localhost/33489] master.HMaster(444): hbase.rootdir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738, hbase.cluster.distributed=false 2023-07-20 16:55:00,955 DEBUG [pool-566-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: INIT 2023-07-20 16:55:00,966 INFO [Listener at localhost/33489] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-20 16:55:00,966 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:55:00,966 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 16:55:00,967 INFO [Listener at localhost/33489] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 16:55:00,967 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:55:00,967 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 16:55:00,967 INFO [Listener at localhost/33489] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 16:55:00,969 INFO [Listener at localhost/33489] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41127 2023-07-20 16:55:00,969 INFO [Listener at localhost/33489] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 16:55:00,971 DEBUG [Listener at localhost/33489] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 16:55:00,972 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:55:00,973 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:55:00,973 INFO [Listener at localhost/33489] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41127 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:55:00,976 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:411270x0, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:55:00,978 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:411270x0, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-20 16:55:00,978 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41127-0x10183a7c131002c connected 2023-07-20 16:55:00,979 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:41127-0x10183a7c131002c, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 16:55:00,979 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:41127-0x10183a7c131002c, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/acl 2023-07-20 16:55:00,986 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41127 2023-07-20 16:55:00,986 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41127 2023-07-20 16:55:00,989 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41127 2023-07-20 16:55:00,989 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41127 2023-07-20 16:55:00,989 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41127 2023-07-20 16:55:00,992 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /1/backup-masters/jenkins-hbase4.apache.org,37313,1689872100929 2023-07-20 16:55:00,994 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-20 16:55:00,994 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/backup-masters/jenkins-hbase4.apache.org,37313,1689872100929 2023-07-20 16:55:00,999 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-20 16:55:00,999 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:41127-0x10183a7c131002c, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/master 2023-07-20 16:55:00,999 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:55:01,000 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-20 16:55:01,002 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/master 2023-07-20 16:55:01,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /1/backup-masters/jenkins-hbase4.apache.org,37313,1689872100929 from backup master directory 2023-07-20 16:55:01,003 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/backup-masters/jenkins-hbase4.apache.org,37313,1689872100929 2023-07-20 16:55:01,003 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/backup-masters 2023-07-20 16:55:01,003 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 16:55:01,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,37313,1689872100929 2023-07-20 16:55:01,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:55:01,052 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x552160d4 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:55:01,055 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6fd9333c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:55:01,055 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 16:55:01,056 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-20 16:55:01,058 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:55:01,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:55:01,066 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/WALs/jenkins-hbase4.apache.org,39639,1689872060057 to hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/WALs/jenkins-hbase4.apache.org,39639,1689872060057-dead as it is dead 2023-07-20 16:55:01,067 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/WALs/jenkins-hbase4.apache.org,39639,1689872060057-dead/jenkins-hbase4.apache.org%2C39639%2C1689872060057.jenkins-hbase4.apache.org%2C39639%2C1689872060057.regiongroup-0.1689872061844 2023-07-20 16:55:01,068 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/WALs/jenkins-hbase4.apache.org,39639,1689872060057-dead/jenkins-hbase4.apache.org%2C39639%2C1689872060057.jenkins-hbase4.apache.org%2C39639%2C1689872060057.regiongroup-0.1689872061844 after 1ms 2023-07-20 16:55:01,068 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/WALs/jenkins-hbase4.apache.org,39639,1689872060057-dead/jenkins-hbase4.apache.org%2C39639%2C1689872060057.jenkins-hbase4.apache.org%2C39639%2C1689872060057.regiongroup-0.1689872061844 to hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C39639%2C1689872060057.jenkins-hbase4.apache.org%2C39639%2C1689872060057.regiongroup-0.1689872061844 2023-07-20 16:55:01,068 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/WALs/jenkins-hbase4.apache.org,39639,1689872060057-dead 2023-07-20 16:55:01,069 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/WALs/jenkins-hbase4.apache.org,37313,1689872100929 2023-07-20 16:55:01,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:55:01,071 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37313%2C1689872100929.jenkins-hbase4.apache.org%2C37313%2C1689872100929.regiongroup-0, suffix=, logDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/WALs/jenkins-hbase4.apache.org,37313,1689872100929, archiveDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/oldWALs, maxLogs=10 2023-07-20 16:55:01,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/WALs/jenkins-hbase4.apache.org,37313,1689872100929/jenkins-hbase4.apache.org%2C37313%2C1689872100929.jenkins-hbase4.apache.org%2C37313%2C1689872100929.regiongroup-0.1689872101071 2023-07-20 16:55:01,090 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40699,DS-82a10023-4ecc-44e1-b86d-cd4749f69898,DISK], DatanodeInfoWithStorage[127.0.0.1:45027,DS-87ab81e2-6016-492e-a423-b308cc471afd,DISK]] 2023-07-20 16:55:01,090 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:55:01,090 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:55:01,091 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:55:01,091 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:55:01,094 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:55:01,096 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-20 16:55:01,096 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-20 16:55:01,104 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/34f5e65d0ce74ea1928444c9b7a76dc3 2023-07-20 16:55:01,104 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:01,105 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-20 16:55:01,105 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C39639%2C1689872060057.jenkins-hbase4.apache.org%2C39639%2C1689872060057.regiongroup-0.1689872061844 2023-07-20 16:55:01,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 155, firstSequenceIdInLog=3, maxSequenceIdInLog=138, path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C39639%2C1689872060057.jenkins-hbase4.apache.org%2C39639%2C1689872060057.regiongroup-0.1689872061844 2023-07-20 16:55:01,111 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C39639%2C1689872060057.jenkins-hbase4.apache.org%2C39639%2C1689872060057.regiongroup-0.1689872061844 2023-07-20 16:55:01,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:55:01,117 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/138.seqid, newMaxSeqId=138, maxSeqId=1 2023-07-20 16:55:01,118 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=139; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59124627, jitterRate=-0.11897440254688263}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 16:55:01,118 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 16:55:01,121 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-20 16:55:01,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-20 16:55:01,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-20 16:55:01,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-20 16:55:01,124 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-20 16:55:01,129 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-20 16:55:01,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-20 16:55:01,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-20 16:55:01,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-20 16:55:01,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-20 16:55:01,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-20 16:55:01,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.replication.AddPeerProcedure 2023-07-20 16:55:01,132 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-20 16:55:01,132 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-20 16:55:01,133 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-20 16:55:01,133 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34183,1689872061357, table=hbase:meta, region=1588230740 2023-07-20 16:55:01,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 2 possibly 'live' servers, and 0 'splitting'. 2023-07-20 16:55:01,137 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,34183,1689872061357 already deleted, retry=false 2023-07-20 16:55:01,138 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,34183,1689872061357 on jenkins-hbase4.apache.org,37313,1689872100929 2023-07-20 16:55:01,144 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,34183,1689872061357, splitWal=true, meta=true 2023-07-20 16:55:01,145 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase4.apache.org,34183,1689872061357 (carryingMeta=true) jenkins-hbase4.apache.org,34183,1689872061357/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@6ed6a550[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-20 16:55:01,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,33103,1689872061305 already deleted, retry=false 2023-07-20 16:55:01,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,33103,1689872061305 on jenkins-hbase4.apache.org,37313,1689872100929 2023-07-20 16:55:01,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,33103,1689872061305, splitWal=true, meta=false 2023-07-20 16:55:01,147 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=19 for jenkins-hbase4.apache.org,33103,1689872061305 (carryingMeta=false) jenkins-hbase4.apache.org,33103,1689872061305/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@6f81196b[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-20 16:55:01,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-20 16:55:01,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-20 16:55:01,162 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/balancer 2023-07-20 16:55:01,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-20 16:55:01,163 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/normalizer 2023-07-20 16:55:01,163 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/split 2023-07-20 16:55:01,164 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/switch/merge 2023-07-20 16:55:01,167 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/snapshot-cleanup 2023-07-20 16:55:01,168 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-20 16:55:01,168 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:41127-0x10183a7c131002c, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/1/running 2023-07-20 16:55:01,168 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:55:01,169 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,37313,1689872100929, sessionid=0x10183a7c131002b, setting cluster-up flag (Was=false) 2023-07-20 16:55:01,176 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/flush-table-proc/acquired, /1/flush-table-proc/reached, /1/flush-table-proc/abort 2023-07-20 16:55:01,177 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37313,1689872100929 2023-07-20 16:55:01,179 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /1/online-snapshot/acquired, /1/online-snapshot/reached, /1/online-snapshot/abort 2023-07-20 16:55:01,180 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37313,1689872100929 2023-07-20 16:55:01,180 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/.hbase-snapshot/.tmp 2023-07-20 16:55:01,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:55:01,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:55:01,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:55:01,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:55:01,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-20 16:55:01,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:01,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 16:55:01,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:01,184 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689872131184 2023-07-20 16:55:01,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-20 16:55:01,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-20 16:55:01,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-20 16:55:01,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-20 16:55:01,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-20 16:55:01,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-20 16:55:01,190 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:01,195 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34183,1689872061357; numProcessing=1 2023-07-20 16:55:01,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-20 16:55:01,195 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34183,1689872061357, splitWal=true, meta=true 2023-07-20 16:55:01,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-20 16:55:01,195 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33103,1689872061305; numProcessing=2 2023-07-20 16:55:01,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-20 16:55:01,195 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=19, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33103,1689872061305, splitWal=true, meta=false 2023-07-20 16:55:01,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-20 16:55:01,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-20 16:55:01,196 INFO [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(951): ClusterId : 258804e0-584b-4fe6-b46a-945d0467644e 2023-07-20 16:55:01,196 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872101196,5,FailOnTimeoutGroup] 2023-07-20 16:55:01,196 DEBUG [RS:0;jenkins-hbase4:41127] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 16:55:01,196 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872101196,5,FailOnTimeoutGroup] 2023-07-20 16:55:01,196 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34183,1689872061357, splitWal=true, meta=true, isMeta: true 2023-07-20 16:55:01,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:01,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-20 16:55:01,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:01,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:01,197 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689872101197, completionTime=-1 2023-07-20 16:55:01,197 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-20 16:55:01,197 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-20 16:55:01,198 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357-splitting 2023-07-20 16:55:01,199 DEBUG [RS:0;jenkins-hbase4:41127] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 16:55:01,199 DEBUG [RS:0;jenkins-hbase4:41127] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 16:55:01,199 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357-splitting dir is empty, no logs to split. 2023-07-20 16:55:01,199 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,34183,1689872061357 WAL count=0, meta=true 2023-07-20 16:55:01,200 DEBUG [RS:0;jenkins-hbase4:41127] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 16:55:01,202 DEBUG [RS:0;jenkins-hbase4:41127] zookeeper.ReadOnlyZKClient(139): Connect 0x5f97caf0 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:55:01,204 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357-splitting dir is empty, no logs to split. 2023-07-20 16:55:01,204 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,34183,1689872061357 WAL count=0, meta=true 2023-07-20 16:55:01,204 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34183,1689872061357 WAL splitting is done? wals=0, meta=true 2023-07-20 16:55:01,205 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-20 16:55:01,208 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-20 16:55:01,210 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 16:55:01,211 DEBUG [RS:0;jenkins-hbase4:41127] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@51503f72, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:55:01,211 DEBUG [RS:0;jenkins-hbase4:41127] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@556fa180, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 16:55:01,219 DEBUG [RS:0;jenkins-hbase4:41127] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:41127 2023-07-20 16:55:01,219 INFO [RS:0;jenkins-hbase4:41127] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 16:55:01,219 INFO [RS:0;jenkins-hbase4:41127] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 16:55:01,219 DEBUG [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 16:55:01,220 INFO [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37313,1689872100929 with isa=jenkins-hbase4.apache.org/172.31.14.131:41127, startcode=1689872100966 2023-07-20 16:55:01,220 DEBUG [RS:0;jenkins-hbase4:41127] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 16:55:01,224 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38231, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 16:55:01,226 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37313] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:01,226 DEBUG [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738 2023-07-20 16:55:01,226 DEBUG [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:38055 2023-07-20 16:55:01,226 DEBUG [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 16:55:01,228 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 16:55:01,229 DEBUG [RS:0;jenkins-hbase4:41127] zookeeper.ZKUtil(162): regionserver:41127-0x10183a7c131002c, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:01,229 WARN [RS:0;jenkins-hbase4:41127] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 16:55:01,229 INFO [RS:0;jenkins-hbase4:41127] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:55:01,229 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41127,1689872100966] 2023-07-20 16:55:01,229 INFO [RS:0;jenkins-hbase4:41127] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:55:01,230 DEBUG [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(1948): logDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:01,245 INFO [RS:0;jenkins-hbase4:41127] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-20 16:55:01,246 DEBUG [RS:0;jenkins-hbase4:41127] zookeeper.ZKUtil(162): regionserver:41127-0x10183a7c131002c, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/rs/jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:01,247 DEBUG [RS:0;jenkins-hbase4:41127] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 16:55:01,248 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=51ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-20 16:55:01,248 INFO [RS:0;jenkins-hbase4:41127] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 16:55:01,249 INFO [RS:0;jenkins-hbase4:41127] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 16:55:01,250 INFO [RS:0;jenkins-hbase4:41127] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 16:55:01,250 INFO [RS:0;jenkins-hbase4:41127] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:01,250 INFO [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 16:55:01,252 INFO [RS:0;jenkins-hbase4:41127] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:01,254 DEBUG [RS:0;jenkins-hbase4:41127] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:01,254 DEBUG [RS:0;jenkins-hbase4:41127] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:01,254 DEBUG [RS:0;jenkins-hbase4:41127] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:01,254 DEBUG [RS:0;jenkins-hbase4:41127] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:01,254 DEBUG [RS:0;jenkins-hbase4:41127] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:01,254 DEBUG [RS:0;jenkins-hbase4:41127] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 16:55:01,254 DEBUG [RS:0;jenkins-hbase4:41127] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:01,254 DEBUG [RS:0;jenkins-hbase4:41127] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:01,254 DEBUG [RS:0;jenkins-hbase4:41127] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:01,254 DEBUG [RS:0;jenkins-hbase4:41127] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:01,258 INFO [RS:0;jenkins-hbase4:41127] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:01,258 INFO [RS:0;jenkins-hbase4:41127] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:01,258 INFO [RS:0;jenkins-hbase4:41127] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:01,269 INFO [RS:0;jenkins-hbase4:41127] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 16:55:01,269 INFO [RS:0;jenkins-hbase4:41127] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41127,1689872100966-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:01,279 INFO [RS:0;jenkins-hbase4:41127] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-20 16:55:01,281 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [1689872080578.syncuptool.replication.org,1234,1] other RSs: [jenkins-hbase4.apache.org,41127,1689872100966] 2023-07-20 16:55:01,290 INFO [RS:0;jenkins-hbase4:41127] regionserver.Replication(203): jenkins-hbase4.apache.org,41127,1689872100966 started 2023-07-20 16:55:01,290 INFO [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,41127,1689872100966, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41127, sessionid=0x10183a7c131002c 2023-07-20 16:55:01,290 DEBUG [RS:0;jenkins-hbase4:41127] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 16:55:01,291 DEBUG [RS:0;jenkins-hbase4:41127] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:01,291 DEBUG [RS:0;jenkins-hbase4:41127] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41127,1689872100966' 2023-07-20 16:55:01,291 DEBUG [RS:0;jenkins-hbase4:41127] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/flush-table-proc/abort' 2023-07-20 16:55:01,291 DEBUG [RS:0;jenkins-hbase4:41127] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/flush-table-proc/acquired' 2023-07-20 16:55:01,291 DEBUG [RS:0;jenkins-hbase4:41127] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 16:55:01,291 DEBUG [RS:0;jenkins-hbase4:41127] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 16:55:01,291 DEBUG [RS:0;jenkins-hbase4:41127] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:01,291 DEBUG [RS:0;jenkins-hbase4:41127] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41127,1689872100966' 2023-07-20 16:55:01,291 DEBUG [RS:0;jenkins-hbase4:41127] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/1/online-snapshot/abort' 2023-07-20 16:55:01,292 DEBUG [RS:0;jenkins-hbase4:41127] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/1/online-snapshot/acquired' 2023-07-20 16:55:01,292 DEBUG [RS:0;jenkins-hbase4:41127] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 16:55:01,292 INFO [RS:0;jenkins-hbase4:41127] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 16:55:01,292 INFO [RS:0;jenkins-hbase4:41127] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 16:55:01,301 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x0ee0425b to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:55:01,306 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@26a3c60f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:55:01,306 INFO [RS:0;jenkins-hbase4:41127.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:55:01,309 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:55:01,313 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10183a7c1310030 connected 2023-07-20 16:55:01,313 INFO [RS:0;jenkins-hbase4:41127.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=258804e0-584b-4fe6-b46a-945d0467644e to cluster=de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:55:01,360 DEBUG [jenkins-hbase4:37313] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-20 16:55:01,361 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41127,1689872100966, state=OPENING 2023-07-20 16:55:01,363 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-20 16:55:01,363 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-20 16:55:01,363 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41127,1689872100966}] 2023-07-20 16:55:01,392 INFO [RS:0;jenkins-hbase4:41127] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:55:01,394 INFO [RS:0;jenkins-hbase4:41127] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0, suffix=, logDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966, archiveDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs, maxLogs=10 2023-07-20 16:55:01,412 DEBUG [RS:0;jenkins-hbase4:41127] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0 for peer 1 2023-07-20 16:55:01,412 INFO [RS:0;jenkins-hbase4:41127] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0.1689872101395 2023-07-20 16:55:01,412 DEBUG [RS:0;jenkins-hbase4:41127] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0 2023-07-20 16:55:01,412 INFO [RS:0;jenkins-hbase4:41127] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 16:55:01,412 DEBUG [RS:0;jenkins-hbase4:41127] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40699,DS-82a10023-4ecc-44e1-b86d-cd4749f69898,DISK], DatanodeInfoWithStorage[127.0.0.1:45027,DS-87ab81e2-6016-492e-a423-b308cc471afd,DISK]] 2023-07-20 16:55:01,412 INFO [RS:0;jenkins-hbase4:41127.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0 2023-07-20 16:55:01,417 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:01,521 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:01,521 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 16:55:01,525 INFO [RS-EventLoopGroup-14-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37868, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 16:55:01,525 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:01,528 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-20 16:55:01,528 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:55:01,528 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:55:01,528 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:55:01,530 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41127%2C1689872100966.meta, suffix=.meta, logDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966, archiveDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs, maxLogs=10 2023-07-20 16:55:01,536 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966/jenkins-hbase4.apache.org%2C41127%2C1689872100966.meta.1689872101530.meta 2023-07-20 16:55:01,536 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45027,DS-87ab81e2-6016-492e-a423-b308cc471afd,DISK], DatanodeInfoWithStorage[127.0.0.1:40699,DS-82a10023-4ecc-44e1-b86d-cd4749f69898,DISK]] 2023-07-20 16:55:01,537 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:55:01,537 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-20 16:55:01,537 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-20 16:55:01,537 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-20 16:55:01,537 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-20 16:55:01,537 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:55:01,537 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-20 16:55:01,537 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-20 16:55:01,539 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-20 16:55:01,540 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/info 2023-07-20 16:55:01,540 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/info 2023-07-20 16:55:01,540 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-20 16:55:01,547 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/info/b7ee159ed6a148b298dac2473952132c 2023-07-20 16:55:01,548 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:01,548 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-20 16:55:01,549 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/rep_barrier 2023-07-20 16:55:01,549 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/rep_barrier 2023-07-20 16:55:01,549 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-20 16:55:01,555 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/rep_barrier/9073089ba3a74d4a8bf812e10f2caad7 2023-07-20 16:55:01,555 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:01,555 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-20 16:55:01,556 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/table 2023-07-20 16:55:01,556 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/table 2023-07-20 16:55:01,557 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-20 16:55:01,562 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/table/b49b27dd474648f69f5e68f6f5886f66 2023-07-20 16:55:01,562 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:01,563 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740 2023-07-20 16:55:01,564 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740 2023-07-20 16:55:01,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-20 16:55:01,567 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-20 16:55:01,568 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61313922, jitterRate=-0.08635136485099792}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-20 16:55:01,568 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-20 16:55:01,571 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=21, masterSystemTime=1689872101521 2023-07-20 16:55:01,575 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-20 16:55:01,576 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-20 16:55:01,576 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41127,1689872100966, state=OPEN 2023-07-20 16:55:01,577 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/meta-region-server 2023-07-20 16:55:01,578 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /1/meta-region-server: CHANGED 2023-07-20 16:55:01,580 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-07-20 16:55:01,580 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41127,1689872100966 in 215 msec 2023-07-20 16:55:01,582 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=18 2023-07-20 16:55:01,582 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 375 msec 2023-07-20 16:55:01,630 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:01,737 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:01,842 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:01,948 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:02,053 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:02,159 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:02,265 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:02,370 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:02,476 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:02,584 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:02,689 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:02,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1554ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-20 16:55:02,795 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:02,900 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:03,006 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:03,111 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:03,216 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:03,321 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:03,427 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:03,532 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:03,637 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:03,743 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:03,848 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:03,953 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:04,059 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:04,165 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:04,253 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3056ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-20 16:55:04,271 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:04,376 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:04,482 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:04,489 INFO [jenkins-hbase4:46817Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:55:04,489 INFO [jenkins-hbase4:46817Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 2 2023-07-20 16:55:04,587 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:04,693 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:04,798 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:04,903 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:05,008 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:05,114 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:05,220 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:05,326 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:05,431 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:05,536 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:05,641 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:05,706 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4509ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-20 16:55:05,706 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-20 16:55:05,710 DEBUG [hconnection-0x68802b61-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:55:05,714 INFO [RS-EventLoopGroup-14-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37880, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:55:05,720 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=eec95f84b91ff86943540982735963f6, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34183,1689872061357, regionLocation=jenkins-hbase4.apache.org,34183,1689872061357, openSeqNum=2 2023-07-20 16:55:05,720 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=ea29850f58e32db8bff448bce6d2160a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33103,1689872061305, regionLocation=jenkins-hbase4.apache.org,33103,1689872061305, openSeqNum=2 2023-07-20 16:55:05,721 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=2627e10c9cf6db0694d7b55799cf6705, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33103,1689872061305, regionLocation=jenkins-hbase4.apache.org,33103,1689872061305, openSeqNum=2 2023-07-20 16:55:05,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-20 16:55:05,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689872165721 2023-07-20 16:55:05,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689872225721 2023-07-20 16:55:05,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 14 msec 2023-07-20 16:55:05,737 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,34183,1689872061357 had 2 regions 2023-07-20 16:55:05,738 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37313,1689872100929-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:05,737 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,33103,1689872061305 had 2 regions 2023-07-20 16:55:05,738 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37313,1689872100929-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:05,738 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37313,1689872100929-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:05,738 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37313, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:05,738 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:05,738 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. is NOT online; state={eec95f84b91ff86943540982735963f6 state=OPEN, ts=1689872105720, server=jenkins-hbase4.apache.org,34183,1689872061357}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-20 16:55:05,739 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33103,1689872061305, splitWal=true, meta=false, isMeta: false 2023-07-20 16:55:05,739 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34183,1689872061357, splitWal=true, meta=true, isMeta: false 2023-07-20 16:55:05,741 DEBUG [PEWorker-3] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305-splitting 2023-07-20 16:55:05,742 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,34183,1689872061357/hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6., unknown_server=jenkins-hbase4.apache.org,33103,1689872061305/t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a., unknown_server=jenkins-hbase4.apache.org,33103,1689872061305/t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:55:05,742 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305-splitting dir is empty, no logs to split. 2023-07-20 16:55:05,743 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,33103,1689872061305 WAL count=0, meta=false 2023-07-20 16:55:05,744 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357-splitting dir is empty, no logs to split. 2023-07-20 16:55:05,744 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,34183,1689872061357 WAL count=0, meta=false 2023-07-20 16:55:05,746 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305-splitting dir is empty, no logs to split. 2023-07-20 16:55:05,746 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,33103,1689872061305 WAL count=0, meta=false 2023-07-20 16:55:05,746 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,33103,1689872061305 WAL splitting is done? wals=0, meta=false 2023-07-20 16:55:05,748 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:05,748 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,34183,1689872061357-splitting dir is empty, no logs to split. 2023-07-20 16:55:05,748 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,34183,1689872061357 WAL count=0, meta=false 2023-07-20 16:55:05,748 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34183,1689872061357 WAL splitting is done? wals=0, meta=false 2023-07-20 16:55:05,750 INFO [PEWorker-3] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,33103,1689872061305 failed, ignore...File hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,33103,1689872061305-splitting does not exist. 2023-07-20 16:55:05,750 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=eec95f84b91ff86943540982735963f6, ASSIGN}] 2023-07-20 16:55:05,751 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=ea29850f58e32db8bff448bce6d2160a, ASSIGN}, {pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=2627e10c9cf6db0694d7b55799cf6705, ASSIGN}] 2023-07-20 16:55:05,751 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=eec95f84b91ff86943540982735963f6, ASSIGN 2023-07-20 16:55:05,752 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=eec95f84b91ff86943540982735963f6, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 16:55:05,752 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=ea29850f58e32db8bff448bce6d2160a, ASSIGN 2023-07-20 16:55:05,752 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=2627e10c9cf6db0694d7b55799cf6705, ASSIGN 2023-07-20 16:55:05,752 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=ea29850f58e32db8bff448bce6d2160a, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 16:55:05,753 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=2627e10c9cf6db0694d7b55799cf6705, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 16:55:05,853 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:05,902 DEBUG [jenkins-hbase4:37313] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-20 16:55:05,905 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=eec95f84b91ff86943540982735963f6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:05,906 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689872105905"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872105905"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872105905"}]},"ts":"1689872105905"} 2023-07-20 16:55:05,906 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=2627e10c9cf6db0694d7b55799cf6705, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:05,906 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=ea29850f58e32db8bff448bce6d2160a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:05,906 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872105906"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872105906"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872105906"}]},"ts":"1689872105906"} 2023-07-20 16:55:05,906 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872105906"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872105906"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872105906"}]},"ts":"1689872105906"} 2023-07-20 16:55:05,909 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=22, state=RUNNABLE; OpenRegionProcedure eec95f84b91ff86943540982735963f6, server=jenkins-hbase4.apache.org,41127,1689872100966}] 2023-07-20 16:55:05,910 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=24, state=RUNNABLE; OpenRegionProcedure 2627e10c9cf6db0694d7b55799cf6705, server=jenkins-hbase4.apache.org,41127,1689872100966}] 2023-07-20 16:55:05,911 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=23, state=RUNNABLE; OpenRegionProcedure ea29850f58e32db8bff448bce6d2160a, server=jenkins-hbase4.apache.org,41127,1689872100966}] 2023-07-20 16:55:05,958 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:06,064 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:06,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:55:06,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:55:06,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1, suffix=, logDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966, archiveDir=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs, maxLogs=10 2023-07-20 16:55:06,070 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:55:06,070 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => eec95f84b91ff86943540982735963f6, NAME => 'hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:55:06,070 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace eec95f84b91ff86943540982735963f6 2023-07-20 16:55:06,071 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:55:06,071 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for eec95f84b91ff86943540982735963f6 2023-07-20 16:55:06,071 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for eec95f84b91ff86943540982735963f6 2023-07-20 16:55:06,072 INFO [StoreOpener-eec95f84b91ff86943540982735963f6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region eec95f84b91ff86943540982735963f6 2023-07-20 16:55:06,075 DEBUG [StoreOpener-eec95f84b91ff86943540982735963f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6/info 2023-07-20 16:55:06,075 DEBUG [StoreOpener-eec95f84b91ff86943540982735963f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6/info 2023-07-20 16:55:06,076 INFO [StoreOpener-eec95f84b91ff86943540982735963f6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eec95f84b91ff86943540982735963f6 columnFamilyName info 2023-07-20 16:55:06,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceManager(623): Start tracking logs for wal group jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1 for peer 1 2023-07-20 16:55:06,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1.1689872106067 2023-07-20 16:55:06,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1 2023-07-20 16:55:06,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 16:55:06,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45027,DS-87ab81e2-6016-492e-a423-b308cc471afd,DISK], DatanodeInfoWithStorage[127.0.0.1:40699,DS-82a10023-4ecc-44e1-b86d-cd4749f69898,DISK]] 2023-07-20 16:55:06,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1 2023-07-20 16:55:06,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2627e10c9cf6db0694d7b55799cf6705, NAME => 't2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:55:06,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup 2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:55:06,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:55:06,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:55:06,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:55:06,087 INFO [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:55:06,091 DEBUG [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/cf1 2023-07-20 16:55:06,091 DEBUG [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/cf1 2023-07-20 16:55:06,091 DEBUG [StoreOpener-eec95f84b91ff86943540982735963f6-1] regionserver.HStore(539): loaded hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6/info/3423509284054478986c16da9315751f 2023-07-20 16:55:06,092 INFO [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2627e10c9cf6db0694d7b55799cf6705 columnFamilyName cf1 2023-07-20 16:55:06,092 INFO [StoreOpener-eec95f84b91ff86943540982735963f6-1] regionserver.HStore(310): Store=eec95f84b91ff86943540982735963f6/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:06,093 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6 2023-07-20 16:55:06,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 83 2023-07-20 16:55:06,094 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6 2023-07-20 16:55:06,099 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for eec95f84b91ff86943540982735963f6 2023-07-20 16:55:06,100 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened eec95f84b91ff86943540982735963f6; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66239752, jitterRate=-0.012950778007507324}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 16:55:06,100 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for eec95f84b91ff86943540982735963f6: 2023-07-20 16:55:06,105 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6., pid=25, masterSystemTime=1689872106061 2023-07-20 16:55:06,106 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f822180116c0458bbb56c3e3d5669ccc 2023-07-20 16:55:06,106 DEBUG [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] regionserver.HStore(539): loaded hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/cf1/f822180116c0458bbb56c3e3d5669ccc 2023-07-20 16:55:06,106 INFO [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] regionserver.HStore(310): Store=2627e10c9cf6db0694d7b55799cf6705/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:06,106 INFO [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:55:06,108 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:55:06,108 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:55:06,109 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=eec95f84b91ff86943540982735963f6, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:06,109 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689872106109"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872106109"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872106109"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872106109"}]},"ts":"1689872106109"} 2023-07-20 16:55:06,111 DEBUG [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/norep 2023-07-20 16:55:06,111 DEBUG [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/norep 2023-07-20 16:55:06,112 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=22 2023-07-20 16:55:06,113 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=22, state=SUCCESS; OpenRegionProcedure eec95f84b91ff86943540982735963f6, server=jenkins-hbase4.apache.org,41127,1689872100966 in 202 msec 2023-07-20 16:55:06,114 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=18 2023-07-20 16:55:06,114 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,34183,1689872061357 after splitting done 2023-07-20 16:55:06,115 INFO [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2627e10c9cf6db0694d7b55799cf6705 columnFamilyName norep 2023-07-20 16:55:06,114 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=eec95f84b91ff86943540982735963f6, ASSIGN in 363 msec 2023-07-20 16:55:06,115 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,34183,1689872061357 from processing; numProcessing=1 2023-07-20 16:55:06,116 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34183,1689872061357, splitWal=true, meta=true in 4.9770 sec 2023-07-20 16:55:06,126 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 685aeda030444458949aeb687cade474 2023-07-20 16:55:06,126 DEBUG [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] regionserver.HStore(539): loaded hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/norep/685aeda030444458949aeb687cade474 2023-07-20 16:55:06,126 INFO [StoreOpener-2627e10c9cf6db0694d7b55799cf6705-1] regionserver.HStore(310): Store=2627e10c9cf6db0694d7b55799cf6705/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:06,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:55:06,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:55:06,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 16:55:06,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:55:06,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2627e10c9cf6db0694d7b55799cf6705; next sequenceid=210; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61656818, jitterRate=-0.08124181628227234}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 16:55:06,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2627e10c9cf6db0694d7b55799cf6705: 2023-07-20 16:55:06,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705., pid=26, masterSystemTime=1689872106061 2023-07-20 16:55:06,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:55:06,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:55:06,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:55:06,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ea29850f58e32db8bff448bce6d2160a, NAME => 't1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:55:06,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:55:06,138 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=2627e10c9cf6db0694d7b55799cf6705, regionState=OPEN, repBarrier=210, openSeqNum=210, regionLocation=jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:06,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:55:06,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:55:06,138 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872106138"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872106138"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872106138"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872106138"}]},"ts":"1689872106138"} 2023-07-20 16:55:06,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:55:06,140 INFO [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:55:06,141 DEBUG [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/cf1 2023-07-20 16:55:06,141 DEBUG [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/cf1 2023-07-20 16:55:06,142 INFO [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea29850f58e32db8bff448bce6d2160a columnFamilyName cf1 2023-07-20 16:55:06,143 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=24 2023-07-20 16:55:06,143 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=24, state=SUCCESS; OpenRegionProcedure 2627e10c9cf6db0694d7b55799cf6705, server=jenkins-hbase4.apache.org,41127,1689872100966 in 230 msec 2023-07-20 16:55:06,145 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=2627e10c9cf6db0694d7b55799cf6705, ASSIGN in 392 msec 2023-07-20 16:55:06,148 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 130b8e810ca245e0ab3ce35f11a7a777 2023-07-20 16:55:06,148 DEBUG [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] regionserver.HStore(539): loaded hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/cf1/130b8e810ca245e0ab3ce35f11a7a777 2023-07-20 16:55:06,148 INFO [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] regionserver.HStore(310): Store=ea29850f58e32db8bff448bce6d2160a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:06,148 INFO [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:55:06,149 DEBUG [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/norep 2023-07-20 16:55:06,150 DEBUG [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/norep 2023-07-20 16:55:06,150 INFO [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea29850f58e32db8bff448bce6d2160a columnFamilyName norep 2023-07-20 16:55:06,156 INFO [StoreFileOpener-norep-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e35537dcd33b41589483da406e2e9bf7 2023-07-20 16:55:06,156 DEBUG [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] regionserver.HStore(539): loaded hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/norep/e35537dcd33b41589483da406e2e9bf7 2023-07-20 16:55:06,156 INFO [StoreOpener-ea29850f58e32db8bff448bce6d2160a-1] regionserver.HStore(310): Store=ea29850f58e32db8bff448bce6d2160a/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:06,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:55:06,158 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:55:06,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 16:55:06,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:55:06,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ea29850f58e32db8bff448bce6d2160a; next sequenceid=110; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63392554, jitterRate=-0.05537733435630798}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 16:55:06,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ea29850f58e32db8bff448bce6d2160a: 2023-07-20 16:55:06,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a., pid=27, masterSystemTime=1689872106061 2023-07-20 16:55:06,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:55:06,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:55:06,168 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=ea29850f58e32db8bff448bce6d2160a, regionState=OPEN, repBarrier=110, openSeqNum=110, regionLocation=jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:06,168 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":6,"row":"t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a.","families":{"rep_barrier":[],"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872106167"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872106167"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872106167"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872106167"}]},"ts":"1689872106167"} 2023-07-20 16:55:06,170 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:06,170 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 16:55:06,170 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:06,170 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0.1689872101395, lastWalPosition=422, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 16:55:06,172 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=23 2023-07-20 16:55:06,172 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=23, state=SUCCESS; OpenRegionProcedure ea29850f58e32db8bff448bce6d2160a, server=jenkins-hbase4.apache.org,41127,1689872100966 in 259 msec 2023-07-20 16:55:06,174 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=19 2023-07-20 16:55:06,174 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,33103,1689872061305 after splitting done 2023-07-20 16:55:06,175 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=ea29850f58e32db8bff448bce6d2160a, ASSIGN in 421 msec 2023-07-20 16:55:06,175 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,33103,1689872061305 from processing; numProcessing=0 2023-07-20 16:55:06,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,33103,1689872061305, splitWal=true, meta=false in 5.0290 sec 2023-07-20 16:55:06,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-20 16:55:06,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 16:55:06,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-20 16:55:06,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1.1689872106067, lastWalPosition=823, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 16:55:06,215 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 16:55:06,277 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:06,290 INFO [jenkins-hbase4:41127Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0]: currently replicating from: hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0.1689872101395 at position: 422 walGroup [jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1]: currently replicating from: hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1.1689872106067 at position: 823 2023-07-20 16:55:06,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-20 16:55:06,382 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:06,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-20 16:55:06,487 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:06,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-20 16:55:06,593 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:06,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-20 16:55:06,698 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:06,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-20 16:55:06,739 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/namespace 2023-07-20 16:55:06,756 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/default 2023-07-20 16:55:06,760 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/1/namespace/hbase 2023-07-20 16:55:06,760 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.757sec 2023-07-20 16:55:06,761 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-20 16:55:06,761 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-20 16:55:06,761 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-20 16:55:06,761 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37313,1689872100929-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-20 16:55:06,761 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37313,1689872100929-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-20 16:55:06,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-20 16:55:06,799 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(139): Connect 0x2edb5a7f to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:55:06,875 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5160e402, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:55:06,877 DEBUG [hconnection-0x67b1ad4c-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:55:06,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-20 16:55:06,879 INFO [RS-EventLoopGroup-14-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37890, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:55:06,881 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:06,885 INFO [Listener at localhost/33489] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-20 16:55:06,885 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2edb5a7f to 127.0.0.1:56032 2023-07-20 16:55:06,885 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:06,886 INFO [Listener at localhost/33489] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:37313 after: jenkins-hbase4.apache.org:37313 2023-07-20 16:55:06,887 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(139): Connect 0x7c074205 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:55:06,891 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@16365b04, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:55:06,891 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x58813696 to 127.0.0.1:56032 2023-07-20 16:55:06,891 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:06,891 DEBUG [Listener at localhost/33489] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-20 16:55:06,891 DEBUG [Listener at localhost/33489] util.JVMClusterUtil(257): Found active master hash=2030539689, stopped=false 2023-07-20 16:55:06,891 INFO [Listener at localhost/33489] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,34355,1689872074166 2023-07-20 16:55:06,894 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:46817-0x10183a7c131001e, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 16:55:06,894 INFO [Listener at localhost/33489] procedure2.ProcedureExecutor(629): Stopping 2023-07-20 16:55:06,894 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 16:55:06,894 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:55:06,894 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7a1d51e8 to 127.0.0.1:56032 2023-07-20 16:55:06,894 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:06,897 INFO [Listener at localhost/33489] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46817,1689872074194' ***** 2023-07-20 16:55:06,897 INFO [Listener at localhost/33489] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 16:55:06,897 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46817-0x10183a7c131001e, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:55:06,897 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:55:06,897 INFO [RS:0;jenkins-hbase4:46817] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 16:55:06,897 INFO [RS:0;jenkins-hbase4:46817] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 16:55:06,897 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 16:55:06,897 INFO [RS:0;jenkins-hbase4:46817] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 16:55:06,899 INFO [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(3305): Received CLOSE for 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:55:06,899 INFO [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(3305): Received CLOSE for c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:55:06,899 INFO [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(3305): Received CLOSE for 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:55:06,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7ebeef24b783c4189ed905a5659b7e20, disabling compactions & flushes 2023-07-20 16:55:06,900 INFO [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:55:06,900 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:55:06,900 DEBUG [RS:0;jenkins-hbase4:46817] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1c0460bd to 127.0.0.1:56032 2023-07-20 16:55:06,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:55:06,900 DEBUG [RS:0;jenkins-hbase4:46817] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:06,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. after waiting 0 ms 2023-07-20 16:55:06,900 INFO [RS:0;jenkins-hbase4:46817] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 16:55:06,900 INFO [RS:0;jenkins-hbase4:46817] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 16:55:06,900 INFO [RS:0;jenkins-hbase4:46817] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 16:55:06,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:55:06,900 INFO [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-20 16:55:06,901 INFO [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-20 16:55:06,901 DEBUG [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(1478): Online Regions={7ebeef24b783c4189ed905a5659b7e20=hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20., c96c52d963776c1249f8d28dc1cee95e=t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e., 1588230740=hbase:meta,,1.1588230740, 058a0bfd9d8bf39d8c91089a151cb277=t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.} 2023-07-20 16:55:06,901 DEBUG [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(1504): Waiting on 058a0bfd9d8bf39d8c91089a151cb277, 1588230740, 7ebeef24b783c4189ed905a5659b7e20, c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:55:06,903 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-20 16:55:06,905 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-20 16:55:06,905 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-20 16:55:06,905 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-20 16:55:06,905 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-20 16:55:06,905 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-20 16:55:06,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-20 16:55:06,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:55:06,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7ebeef24b783c4189ed905a5659b7e20: 2023-07-20 16:55:06,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:55:06,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c96c52d963776c1249f8d28dc1cee95e, disabling compactions & flushes 2023-07-20 16:55:06,914 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:55:06,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:55:06,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. after waiting 0 ms 2023-07-20 16:55:06,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:55:06,915 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing c96c52d963776c1249f8d28dc1cee95e 2/2 column families, dataSize=2.72 KB heapSize=10.66 KB 2023-07-20 16:55:06,932 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/.tmp/info/1bbd25e4897c48f2a459de39ee118f1a 2023-07-20 16:55:06,945 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/.tmp/info/1bbd25e4897c48f2a459de39ee118f1a as hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info/1bbd25e4897c48f2a459de39ee118f1a 2023-07-20 16:55:06,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=15 (bloomFilter=true), to=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/.tmp/cf1/34a7703742324ea7806222645495771e 2023-07-20 16:55:06,953 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info/1bbd25e4897c48f2a459de39ee118f1a, entries=24, sequenceid=31, filesize=7.4 K 2023-07-20 16:55:06,955 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 50ms, sequenceid=31, compaction requested=false 2023-07-20 16:55:06,959 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 34a7703742324ea7806222645495771e 2023-07-20 16:55:06,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/.tmp/cf1/34a7703742324ea7806222645495771e as hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1/34a7703742324ea7806222645495771e 2023-07-20 16:55:06,973 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 34a7703742324ea7806222645495771e 2023-07-20 16:55:06,973 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1/34a7703742324ea7806222645495771e, entries=100, sequenceid=15, filesize=7.9 K 2023-07-20 16:55:06,974 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-20 16:55:06,974 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2790, heapSize ~10.39 KB/10640, currentSize=0 B/0 for c96c52d963776c1249f8d28dc1cee95e in 60ms, sequenceid=15, compaction requested=false 2023-07-20 16:55:06,976 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-20 16:55:06,984 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-20 16:55:06,984 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-20 16:55:06,984 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-20 16:55:06,991 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:55:06,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-20 16:55:07,001 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:07,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/recovered.edits/18.seqid, newMaxSeqId=18, maxSeqId=11 2023-07-20 16:55:07,004 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:55:07,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c96c52d963776c1249f8d28dc1cee95e: 2023-07-20 16:55:07,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:55:07,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 058a0bfd9d8bf39d8c91089a151cb277, disabling compactions & flushes 2023-07-20 16:55:07,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:55:07,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:55:07,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. after waiting 0 ms 2023-07-20 16:55:07,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:55:07,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 058a0bfd9d8bf39d8c91089a151cb277 2/2 column families, dataSize=1.36 KB heapSize=5.58 KB 2023-07-20 16:55:07,021 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.36 KB at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/.tmp/cf1/cb4f45b9e6a94e068dda624387ebacad 2023-07-20 16:55:07,027 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb4f45b9e6a94e068dda624387ebacad 2023-07-20 16:55:07,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/.tmp/cf1/cb4f45b9e6a94e068dda624387ebacad as hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1/cb4f45b9e6a94e068dda624387ebacad 2023-07-20 16:55:07,035 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb4f45b9e6a94e068dda624387ebacad 2023-07-20 16:55:07,035 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1/cb4f45b9e6a94e068dda624387ebacad, entries=50, sequenceid=14, filesize=6.4 K 2023-07-20 16:55:07,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.36 KB/1390, heapSize ~5.31 KB/5440, currentSize=0 B/0 for 058a0bfd9d8bf39d8c91089a151cb277 in 31ms, sequenceid=14, compaction requested=false 2023-07-20 16:55:07,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=10 2023-07-20 16:55:07,051 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:55:07,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 058a0bfd9d8bf39d8c91089a151cb277: 2023-07-20 16:55:07,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:55:07,101 INFO [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46817,1689872074194; all regions closed. 2023-07-20 16:55:07,102 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:55:07,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-20 16:55:07,112 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:07,113 DEBUG [RS:0;jenkins-hbase4:46817] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs 2023-07-20 16:55:07,113 INFO [RS:0;jenkins-hbase4:46817] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C46817%2C1689872074194.meta:.meta(num 1689872074744) 2023-07-20 16:55:07,113 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:55:07,121 DEBUG [RS:0;jenkins-hbase4:46817] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs 2023-07-20 16:55:07,121 INFO [RS:0;jenkins-hbase4:46817] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C46817%2C1689872074194.jenkins-hbase4.apache.org%2C46817%2C1689872074194.regiongroup-0:(num 1689872074594) 2023-07-20 16:55:07,121 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:55:07,138 DEBUG [RS:0;jenkins-hbase4:46817] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs 2023-07-20 16:55:07,138 INFO [RS:0;jenkins-hbase4:46817] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C46817%2C1689872074194.jenkins-hbase4.apache.org%2C46817%2C1689872074194.regiongroup-1:(num 1689872079157) 2023-07-20 16:55:07,138 DEBUG [RS:0;jenkins-hbase4:46817] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:07,138 INFO [RS:0;jenkins-hbase4:46817] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:55:07,138 INFO [RS:0;jenkins-hbase4:46817] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-20 16:55:07,139 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 16:55:07,139 DEBUG [RS:0;jenkins-hbase4:46817] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5aa9a029 to 127.0.0.1:56032 2023-07-20 16:55:07,139 DEBUG [RS:0;jenkins-hbase4:46817] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:07,140 INFO [RS:0;jenkins-hbase4:46817] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46817 2023-07-20 16:55:07,143 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:46817-0x10183a7c131001e, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,46817,1689872074194 2023-07-20 16:55:07,143 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:55:07,143 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:46817-0x10183a7c131001e, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:55:07,143 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310027, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:55:07,143 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 16:55:07,144 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46817,1689872074194] 2023-07-20 16:55:07,144 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46817,1689872074194; numProcessing=1 2023-07-20 16:55:07,145 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46817,1689872074194 already deleted, retry=false 2023-07-20 16:55:07,145 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46817,1689872074194 expired; onlineServers=0 2023-07-20 16:55:07,146 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34355,1689872074166' ***** 2023-07-20 16:55:07,146 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-20 16:55:07,147 DEBUG [M:0;jenkins-hbase4:34355] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7d380851, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 16:55:07,147 INFO [M:0;jenkins-hbase4:34355] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34355,1689872074166 2023-07-20 16:55:07,147 INFO [M:0;jenkins-hbase4:34355] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34355,1689872074166; all regions closed. 2023-07-20 16:55:07,147 DEBUG [M:0;jenkins-hbase4:34355] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:07,147 DEBUG [M:0;jenkins-hbase4:34355] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-20 16:55:07,147 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-20 16:55:07,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872074407] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872074407,5,FailOnTimeoutGroup] 2023-07-20 16:55:07,147 DEBUG [M:0;jenkins-hbase4:34355] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-20 16:55:07,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872074408] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872074408,5,FailOnTimeoutGroup] 2023-07-20 16:55:07,147 INFO [M:0;jenkins-hbase4:34355] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-20 16:55:07,148 INFO [M:0;jenkins-hbase4:34355] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-20 16:55:07,148 INFO [M:0;jenkins-hbase4:34355] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-20 16:55:07,148 DEBUG [M:0;jenkins-hbase4:34355] master.HMaster(1512): Stopping service threads 2023-07-20 16:55:07,149 INFO [M:0;jenkins-hbase4:34355] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-20 16:55:07,149 ERROR [M:0;jenkins-hbase4:34355] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-20 16:55:07,150 INFO [M:0;jenkins-hbase4:34355] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-20 16:55:07,150 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-20 16:55:07,152 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-20 16:55:07,152 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:55:07,152 DEBUG [M:0;jenkins-hbase4:34355] zookeeper.RecoverableZooKeeper(172): Node /2/master already deleted, retry=false 2023-07-20 16:55:07,152 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 16:55:07,152 DEBUG [M:0;jenkins-hbase4:34355] master.ActiveMasterManager(335): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Failed delete of our master address node; KeeperErrorCode = NoNode for /2/master 2023-07-20 16:55:07,153 INFO [M:0;jenkins-hbase4:34355] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-20 16:55:07,154 INFO [M:0;jenkins-hbase4:34355] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-20 16:55:07,154 DEBUG [M:0;jenkins-hbase4:34355] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-20 16:55:07,154 INFO [M:0;jenkins-hbase4:34355] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:55:07,154 DEBUG [M:0;jenkins-hbase4:34355] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:55:07,154 DEBUG [M:0;jenkins-hbase4:34355] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-20 16:55:07,154 DEBUG [M:0;jenkins-hbase4:34355] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:55:07,155 INFO [M:0;jenkins-hbase4:34355] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=36.70 KB heapSize=47.16 KB 2023-07-20 16:55:07,172 INFO [M:0;jenkins-hbase4:34355] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=36.70 KB at sequenceid=239 (bloomFilter=true), to=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3b9a23bbf9b647e9b1922993c9037ba7 2023-07-20 16:55:07,177 INFO [M:0;jenkins-hbase4:34355] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b9a23bbf9b647e9b1922993c9037ba7 2023-07-20 16:55:07,178 DEBUG [M:0;jenkins-hbase4:34355] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3b9a23bbf9b647e9b1922993c9037ba7 as hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3b9a23bbf9b647e9b1922993c9037ba7 2023-07-20 16:55:07,183 INFO [M:0;jenkins-hbase4:34355] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b9a23bbf9b647e9b1922993c9037ba7 2023-07-20 16:55:07,183 INFO [M:0;jenkins-hbase4:34355] regionserver.HStore(1080): Added hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3b9a23bbf9b647e9b1922993c9037ba7, entries=20, sequenceid=239, filesize=5.6 K 2023-07-20 16:55:07,184 INFO [M:0;jenkins-hbase4:34355] regionserver.HRegion(2948): Finished flush of dataSize ~36.70 KB/37578, heapSize ~47.15 KB/48280, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 29ms, sequenceid=239, compaction requested=false 2023-07-20 16:55:07,187 INFO [M:0;jenkins-hbase4:34355] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:55:07,187 DEBUG [M:0;jenkins-hbase4:34355] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 16:55:07,187 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,34355,1689872074166 2023-07-20 16:55:07,197 INFO [M:0;jenkins-hbase4:34355] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-20 16:55:07,197 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 16:55:07,197 INFO [M:0;jenkins-hbase4:34355] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34355 2023-07-20 16:55:07,200 DEBUG [M:0;jenkins-hbase4:34355] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,34355,1689872074166 already deleted, retry=false 2023-07-20 16:55:07,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-20 16:55:07,216 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:07,321 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:07,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-20 16:55:07,391 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:07,391 INFO [M:0;jenkins-hbase4:34355] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34355,1689872074166; zookeeper connection closed. 2023-07-20 16:55:07,391 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:34355-0x10183a7c131001d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:07,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 823 2023-07-20 16:55:07,427 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:07,492 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:46817-0x10183a7c131001e, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:07,492 INFO [RS:0;jenkins-hbase4:46817] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46817,1689872074194; zookeeper connection closed. 2023-07-20 16:55:07,492 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:46817-0x10183a7c131001e, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:07,493 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@158d7e61] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@158d7e61 2023-07-20 16:55:07,493 INFO [Listener at localhost/33489] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-20 16:55:07,495 DEBUG [hconnection-0x22faeb39-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:55:07,499 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37896, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:55:07,534 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:07,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 3119 2023-07-20 16:55:07,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 22 WAL entries eligible for replication 2023-07-20 16:55:07,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 3119 2023-07-20 16:55:07,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/ea29850f58e32db8bff448bce6d2160a/112=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/113=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/114=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/115=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/116=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/117=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/118=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/119=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/120=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/121=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/122=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/123=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/124=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/125=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/126=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/127=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/128=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/129=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/130=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/131=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/132=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/133=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1.1689872106067, lastWalPosition=3119, nbRowKeys=22, nbHFiles=0, heapSize=4576, lastSeqIds={}, endOfFile=false,usedBufferSize=4576] 2023-07-20 16:55:07,539 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-20 16:55:07,640 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-20 16:55:07,641 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:07,657 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14235 2023-07-20 16:55:07,657 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 104 WAL entries eligible for replication 2023-07-20 16:55:07,657 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 14235 2023-07-20 16:55:07,747 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:07,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] wal.ProtobufLogReader(420): EOF at position 26183 2023-07-20 16:55:07,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] regionserver.ReplicationSourceWALReader(162): Read 112 WAL entries eligible for replication 2023-07-20 16:55:07,841 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7c074205 to 127.0.0.1:56032 2023-07-20 16:55:07,841 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:07,841 DEBUG [Listener at localhost/33489] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-20 16:55:07,841 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] regionserver.ReplicationSinkManager(154): No sinks available at peer. Will not be able to replicate 2023-07-20 16:55:07,841 DEBUG [Listener at localhost/33489] util.JVMClusterUtil(257): Found active master hash=1396003883, stopped=false 2023-07-20 16:55:07,841 INFO [Listener at localhost/33489] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,37313,1689872100929 2023-07-20 16:55:07,844 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:41127-0x10183a7c131002c, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-20 16:55:07,844 INFO [Listener at localhost/33489] procedure2.ProcedureExecutor(629): Stopping 2023-07-20 16:55:07,844 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/running 2023-07-20 16:55:07,844 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:55:07,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41127-0x10183a7c131002c, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 16:55:07,846 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x552160d4 to 127.0.0.1:56032 2023-07-20 16:55:07,847 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:07,847 INFO [Listener at localhost/33489] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41127,1689872100966' ***** 2023-07-20 16:55:07,847 INFO [Listener at localhost/33489] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 16:55:07,847 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/running 2023-07-20 16:55:07,848 INFO [RS:0;jenkins-hbase4:41127] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 16:55:07,848 INFO [RS:0;jenkins-hbase4:41127] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 16:55:07,848 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 16:55:07,848 INFO [RS:0;jenkins-hbase4:41127] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 16:55:07,849 INFO [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(3305): Received CLOSE for 2627e10c9cf6db0694d7b55799cf6705 2023-07-20 16:55:07,849 INFO [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(3305): Received CLOSE for eec95f84b91ff86943540982735963f6 2023-07-20 16:55:07,849 INFO [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(3305): Received CLOSE for ea29850f58e32db8bff448bce6d2160a 2023-07-20 16:55:07,850 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2627e10c9cf6db0694d7b55799cf6705, disabling compactions & flushes 2023-07-20 16:55:07,850 INFO [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:07,850 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:55:07,850 DEBUG [RS:0;jenkins-hbase4:41127] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5f97caf0 to 127.0.0.1:56032 2023-07-20 16:55:07,850 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:55:07,851 DEBUG [RS:0;jenkins-hbase4:41127] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:07,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. after waiting 0 ms 2023-07-20 16:55:07,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:55:07,851 INFO [RS:0;jenkins-hbase4:41127] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 16:55:07,851 INFO [RS:0;jenkins-hbase4:41127] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 16:55:07,851 INFO [RS:0;jenkins-hbase4:41127] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 16:55:07,851 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 2627e10c9cf6db0694d7b55799cf6705 2/2 column families, dataSize=7.05 KB heapSize=22.49 KB 2023-07-20 16:55:07,851 INFO [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-20 16:55:07,851 INFO [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-20 16:55:07,851 DEBUG [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, 2627e10c9cf6db0694d7b55799cf6705=t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705., eec95f84b91ff86943540982735963f6=hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6., ea29850f58e32db8bff448bce6d2160a=t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a.} 2023-07-20 16:55:07,852 DEBUG [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(1504): Waiting on 1588230740, 2627e10c9cf6db0694d7b55799cf6705, ea29850f58e32db8bff448bce6d2160a, eec95f84b91ff86943540982735963f6 2023-07-20 16:55:07,852 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-20 16:55:07,853 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-20 16:55:07,853 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-20 16:55:07,853 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-20 16:55:07,853 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-20 16:55:07,853 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.94 KB heapSize=5.62 KB 2023-07-20 16:55:07,856 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 422 2023-07-20 16:55:07,868 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:55:07,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/.tmp/cf1/a9f9db97cf944a88af9c5b6daf0a1ed0 2023-07-20 16:55:07,882 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/.tmp/info/500a30f7e5f647ccad45b8dd48cdbdc5 2023-07-20 16:55:07,901 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=413 (bloomFilter=true), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/.tmp/norep/b512a68edecd4c95bd269e37b0f31538 2023-07-20 16:55:07,905 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226 B at sequenceid=31 (bloomFilter=false), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/.tmp/rep_barrier/396d26fbbb794113a1e827b6967feed9 2023-07-20 16:55:07,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/.tmp/cf1/a9f9db97cf944a88af9c5b6daf0a1ed0 as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/cf1/a9f9db97cf944a88af9c5b6daf0a1ed0 2023-07-20 16:55:07,910 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/.tmp/info/500a30f7e5f647ccad45b8dd48cdbdc5 as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/info/500a30f7e5f647ccad45b8dd48cdbdc5 2023-07-20 16:55:07,916 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/cf1/a9f9db97cf944a88af9c5b6daf0a1ed0, entries=200, sequenceid=413, filesize=12.5 K 2023-07-20 16:55:07,917 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/info/500a30f7e5f647ccad45b8dd48cdbdc5, entries=24, sequenceid=31, filesize=7.4 K 2023-07-20 16:55:07,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/.tmp/norep/b512a68edecd4c95bd269e37b0f31538 as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/norep/b512a68edecd4c95bd269e37b0f31538 2023-07-20 16:55:07,918 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/.tmp/rep_barrier/396d26fbbb794113a1e827b6967feed9 as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/rep_barrier/396d26fbbb794113a1e827b6967feed9 2023-07-20 16:55:07,924 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/norep/b512a68edecd4c95bd269e37b0f31538, entries=1, sequenceid=413, filesize=4.8 K 2023-07-20 16:55:07,925 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/rep_barrier/396d26fbbb794113a1e827b6967feed9, entries=2, sequenceid=31, filesize=4.9 K 2023-07-20 16:55:07,926 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.05 KB/7221, heapSize ~22.46 KB/23000, currentSize=0 B/0 for 2627e10c9cf6db0694d7b55799cf6705 in 75ms, sequenceid=413, compaction requested=false 2023-07-20 16:55:07,926 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.94 KB/3012, heapSize ~5.34 KB/5464, currentSize=0 B/0 for 1588230740 in 73ms, sequenceid=31, compaction requested=false 2023-07-20 16:55:07,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t2_syncup/2627e10c9cf6db0694d7b55799cf6705/recovered.edits/416.seqid, newMaxSeqId=416, maxSeqId=209 2023-07-20 16:55:07,950 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/meta/1588230740/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-07-20 16:55:07,951 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:55:07,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2627e10c9cf6db0694d7b55799cf6705: 2023-07-20 16:55:07,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689872067354.2627e10c9cf6db0694d7b55799cf6705. 2023-07-20 16:55:07,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing eec95f84b91ff86943540982735963f6, disabling compactions & flushes 2023-07-20 16:55:07,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:55:07,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:55:07,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. after waiting 0 ms 2023-07-20 16:55:07,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:55:07,953 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-20 16:55:07,953 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-20 16:55:07,953 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-20 16:55:07,953 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-20 16:55:07,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/hbase/namespace/eec95f84b91ff86943540982735963f6/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-07-20 16:55:07,960 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:55:07,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for eec95f84b91ff86943540982735963f6: 2023-07-20 16:55:07,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689872063202.eec95f84b91ff86943540982735963f6. 2023-07-20 16:55:07,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ea29850f58e32db8bff448bce6d2160a, disabling compactions & flushes 2023-07-20 16:55:07,961 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:55:07,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:55:07,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. after waiting 0 ms 2023-07-20 16:55:07,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:55:07,961 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ea29850f58e32db8bff448bce6d2160a 2/2 column families, dataSize=3.44 KB heapSize=11.55 KB 2023-07-20 16:55:07,963 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-20 16:55:07,963 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 16:55:07,963 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.ProtobufLogReader(420): EOF at position 728 2023-07-20 16:55:07,963 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] regionserver.ReplicationSourceShipper(110): Shipper from source 1 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0.1689872101395, lastWalPosition=728, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 16:55:07,973 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/.tmp/cf1/4ba679396649433a952111f5c1e33e14 2023-07-20 16:55:07,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=41 B at sequenceid=213 (bloomFilter=true), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/.tmp/norep/d6a3744e0f074f23983ebc390c7f1adf 2023-07-20 16:55:07,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/.tmp/cf1/4ba679396649433a952111f5c1e33e14 as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/cf1/4ba679396649433a952111f5c1e33e14 2023-07-20 16:55:07,998 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/cf1/4ba679396649433a952111f5c1e33e14, entries=100, sequenceid=213, filesize=8.4 K 2023-07-20 16:55:07,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/.tmp/norep/d6a3744e0f074f23983ebc390c7f1adf as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/norep/d6a3744e0f074f23983ebc390c7f1adf 2023-07-20 16:55:08,004 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/norep/d6a3744e0f074f23983ebc390c7f1adf, entries=1, sequenceid=213, filesize=4.8 K 2023-07-20 16:55:08,005 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.44 KB/3521, heapSize ~11.52 KB/11800, currentSize=0 B/0 for ea29850f58e32db8bff448bce6d2160a in 44ms, sequenceid=213, compaction requested=false 2023-07-20 16:55:08,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/data/default/t1_syncup/ea29850f58e32db8bff448bce6d2160a/recovered.edits/216.seqid, newMaxSeqId=216, maxSeqId=109 2023-07-20 16:55:08,015 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:55:08,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ea29850f58e32db8bff448bce6d2160a: 2023-07-20 16:55:08,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689872066710.ea29850f58e32db8bff448bce6d2160a. 2023-07-20 16:55:08,052 INFO [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41127,1689872100966; all regions closed. 2023-07-20 16:55:08,053 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:08,059 DEBUG [RS:0;jenkins-hbase4:41127] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs 2023-07-20 16:55:08,059 INFO [RS:0;jenkins-hbase4:41127] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C41127%2C1689872100966.meta:.meta(num 1689872101530) 2023-07-20 16:55:08,060 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:08,065 DEBUG [RS:0;jenkins-hbase4:41127] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs 2023-07-20 16:55:08,065 INFO [RS:0;jenkins-hbase4:41127] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0:(num 1689872101395) 2023-07-20 16:55:08,066 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:08,071 INFO [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] wal.AbstractFSWALProvider(464): Log hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/WALs/jenkins-hbase4.apache.org,41127,1689872100966/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0.1689872101395 was moved to hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0.1689872101395 2023-07-20 16:55:08,071 DEBUG [RS:0;jenkins-hbase4:41127] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs 2023-07-20 16:55:08,071 INFO [RS:0;jenkins-hbase4:41127] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1:(num 1689872106067) 2023-07-20 16:55:08,071 DEBUG [RS:0;jenkins-hbase4:41127] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:08,071 INFO [RS:0;jenkins-hbase4:41127] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:55:08,072 INFO [RS:0;jenkins-hbase4:41127] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-20 16:55:08,072 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 16:55:08,072 INFO [RS:0;jenkins-hbase4:41127] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-20 16:55:08,074 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0.1689872101395 2023-07-20 16:55:08,074 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1] regionserver.ReplicationSourceWALReader(162): Read 0 WAL entries eligible for replication 2023-07-20 16:55:08,141 WARN [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1] regionserver.HBaseInterClusterReplicationEndpoint(515): No replication sinks found, returning without replicating. The source should retry with the same set of edits. Not logging this again for the next 10 seconds. 2023-07-20 16:55:08,246 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310030, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:08,247 DEBUG [RS:0;jenkins-hbase4:41127] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0ee0425b to 127.0.0.1:56032 2023-07-20 16:55:08,247 DEBUG [RS:0;jenkins-hbase4:41127.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310030, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:08,247 DEBUG [RS:0;jenkins-hbase4:41127] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:08,348 INFO [RS:0;jenkins-hbase4:41127] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS:0;jenkins-hbase4:41127.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1 terminated 2023-07-20 16:55:08,349 INFO [RS:0;jenkins-hbase4:41127] regionserver.ReplicationSource(710): peerId=1, ReplicationSourceWorker RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1 terminated 2023-07-20 16:55:08,351 INFO [RS:0;jenkins-hbase4:41127] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41127 2023-07-20 16:55:08,355 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:41127-0x10183a7c131002c, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/rs/jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:08,355 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 16:55:08,355 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:41127-0x10183a7c131002c, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/rs 2023-07-20 16:55:08,357 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41127,1689872100966] 2023-07-20 16:55:08,357 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41127,1689872100966; numProcessing=1 2023-07-20 16:55:08,358 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /1/draining/jenkins-hbase4.apache.org,41127,1689872100966 already deleted, retry=false 2023-07-20 16:55:08,358 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41127,1689872100966 expired; onlineServers=0 2023-07-20 16:55:08,358 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,37313,1689872100929' ***** 2023-07-20 16:55:08,358 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-20 16:55:08,359 DEBUG [M:0;jenkins-hbase4:37313] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@570aca95, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 16:55:08,359 INFO [M:0;jenkins-hbase4:37313] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,37313,1689872100929 2023-07-20 16:55:08,359 INFO [M:0;jenkins-hbase4:37313] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,37313,1689872100929; all regions closed. 2023-07-20 16:55:08,359 DEBUG [M:0;jenkins-hbase4:37313] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:08,359 DEBUG [M:0;jenkins-hbase4:37313] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-20 16:55:08,359 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-20 16:55:08,359 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872101196] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872101196,5,FailOnTimeoutGroup] 2023-07-20 16:55:08,359 DEBUG [M:0;jenkins-hbase4:37313] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-20 16:55:08,359 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872101196] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872101196,5,FailOnTimeoutGroup] 2023-07-20 16:55:08,360 INFO [M:0;jenkins-hbase4:37313] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-20 16:55:08,361 INFO [M:0;jenkins-hbase4:37313] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-20 16:55:08,361 INFO [M:0;jenkins-hbase4:37313] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-20 16:55:08,361 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/master 2023-07-20 16:55:08,361 DEBUG [M:0;jenkins-hbase4:37313] master.HMaster(1512): Stopping service threads 2023-07-20 16:55:08,361 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1 2023-07-20 16:55:08,361 INFO [M:0;jenkins-hbase4:37313] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-20 16:55:08,362 ERROR [M:0;jenkins-hbase4:37313] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-20 16:55:08,362 INFO [M:0;jenkins-hbase4:37313] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-20 16:55:08,362 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on znode that does not yet exist, /1/master 2023-07-20 16:55:08,362 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-20 16:55:08,362 DEBUG [M:0;jenkins-hbase4:37313] zookeeper.ZKUtil(398): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Unable to get data of znode /1/master because node does not exist (not an error) 2023-07-20 16:55:08,362 WARN [M:0;jenkins-hbase4:37313] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-20 16:55:08,362 INFO [M:0;jenkins-hbase4:37313] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-20 16:55:08,363 INFO [M:0;jenkins-hbase4:37313] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-20 16:55:08,364 DEBUG [M:0;jenkins-hbase4:37313] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-20 16:55:08,364 INFO [M:0;jenkins-hbase4:37313] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:55:08,365 DEBUG [M:0;jenkins-hbase4:37313] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:55:08,365 DEBUG [M:0;jenkins-hbase4:37313] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-20 16:55:08,365 DEBUG [M:0;jenkins-hbase4:37313] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:55:08,365 INFO [M:0;jenkins-hbase4:37313] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=29.71 KB heapSize=36.94 KB 2023-07-20 16:55:08,376 INFO [M:0;jenkins-hbase4:37313] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=29.71 KB at sequenceid=218 (bloomFilter=true), to=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/dafc6c3edf6548eea0eebab1ddd7fb1a 2023-07-20 16:55:08,381 DEBUG [M:0;jenkins-hbase4:37313] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/dafc6c3edf6548eea0eebab1ddd7fb1a as hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/dafc6c3edf6548eea0eebab1ddd7fb1a 2023-07-20 16:55:08,386 INFO [M:0;jenkins-hbase4:37313] regionserver.HStore(1080): Added hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/dafc6c3edf6548eea0eebab1ddd7fb1a, entries=10, sequenceid=218, filesize=5.8 K 2023-07-20 16:55:08,387 INFO [M:0;jenkins-hbase4:37313] regionserver.HRegion(2948): Finished flush of dataSize ~29.71 KB/30420, heapSize ~36.92 KB/37808, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 22ms, sequenceid=218, compaction requested=false 2023-07-20 16:55:08,390 INFO [M:0;jenkins-hbase4:37313] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:55:08,390 DEBUG [M:0;jenkins-hbase4:37313] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 16:55:08,390 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/MasterData/WALs/jenkins-hbase4.apache.org,37313,1689872100929 2023-07-20 16:55:08,395 INFO [M:0;jenkins-hbase4:37313] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-20 16:55:08,395 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 16:55:08,396 INFO [M:0;jenkins-hbase4:37313] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37313 2023-07-20 16:55:08,398 DEBUG [M:0;jenkins-hbase4:37313] zookeeper.RecoverableZooKeeper(172): Node /1/rs/jenkins-hbase4.apache.org,37313,1689872100929 already deleted, retry=false 2023-07-20 16:55:08,457 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:41127-0x10183a7c131002c, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:08,457 INFO [RS:0;jenkins-hbase4:41127] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41127,1689872100966; zookeeper connection closed. 2023-07-20 16:55:08,457 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:41127-0x10183a7c131002c, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:08,458 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@27e6b8f3] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@27e6b8f3 2023-07-20 16:55:08,458 INFO [Listener at localhost/33489] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-20 16:55:08,557 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:08,557 INFO [M:0;jenkins-hbase4:37313] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,37313,1689872100929; zookeeper connection closed. 2023-07-20 16:55:08,557 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:37313-0x10183a7c131002b, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:08,560 INFO [Listener at localhost/33489] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 16:55:08,560 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:55:08,561 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 16:55:08,561 INFO [Listener at localhost/33489] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 16:55:08,561 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:55:08,561 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 16:55:08,561 INFO [Listener at localhost/33489] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 16:55:08,562 INFO [Listener at localhost/33489] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39055 2023-07-20 16:55:08,563 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:55:08,564 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:55:08,564 INFO [Listener at localhost/33489] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39055 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:55:08,567 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:390550x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:55:08,567 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:39055-0x10183a7c1310033 connected 2023-07-20 16:55:08,569 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 16:55:08,570 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:55:08,570 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 16:55:08,570 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39055 2023-07-20 16:55:08,571 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39055 2023-07-20 16:55:08,571 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39055 2023-07-20 16:55:08,571 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39055 2023-07-20 16:55:08,571 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39055 2023-07-20 16:55:08,571 INFO [Listener at localhost/33489] master.HMaster(444): hbase.rootdir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a, hbase.cluster.distributed=false 2023-07-20 16:55:08,572 DEBUG [pool-668-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: INIT 2023-07-20 16:55:08,583 INFO [Listener at localhost/33489] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=18 2023-07-20 16:55:08,584 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:55:08,584 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-20 16:55:08,584 INFO [Listener at localhost/33489] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-20 16:55:08,584 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-20 16:55:08,584 INFO [Listener at localhost/33489] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-20 16:55:08,584 INFO [Listener at localhost/33489] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-20 16:55:08,585 INFO [Listener at localhost/33489] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44889 2023-07-20 16:55:08,586 INFO [Listener at localhost/33489] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-20 16:55:08,587 DEBUG [Listener at localhost/33489] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-20 16:55:08,587 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:55:08,588 INFO [Listener at localhost/33489] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:55:08,589 INFO [Listener at localhost/33489] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44889 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:55:08,591 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:448890x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:55:08,593 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:448890x0, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 16:55:08,593 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:44889-0x10183a7c1310034 connected 2023-07-20 16:55:08,593 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:44889-0x10183a7c1310034, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:55:08,594 DEBUG [Listener at localhost/33489] zookeeper.ZKUtil(164): regionserver:44889-0x10183a7c1310034, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/acl 2023-07-20 16:55:08,594 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44889 2023-07-20 16:55:08,598 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44889 2023-07-20 16:55:08,599 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44889 2023-07-20 16:55:08,599 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44889 2023-07-20 16:55:08,601 DEBUG [Listener at localhost/33489] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44889 2023-07-20 16:55:08,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /2/backup-masters/jenkins-hbase4.apache.org,39055,1689872108560 2023-07-20 16:55:08,604 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-20 16:55:08,605 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/backup-masters/jenkins-hbase4.apache.org,39055,1689872108560 2023-07-20 16:55:08,607 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 16:55:08,607 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:44889-0x10183a7c1310034, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/master 2023-07-20 16:55:08,608 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:55:08,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-20 16:55:08,610 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/master 2023-07-20 16:55:08,610 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /2/backup-masters/jenkins-hbase4.apache.org,39055,1689872108560 from backup master directory 2023-07-20 16:55:08,611 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/backup-masters/jenkins-hbase4.apache.org,39055,1689872108560 2023-07-20 16:55:08,611 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/backup-masters 2023-07-20 16:55:08,611 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 16:55:08,611 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,39055,1689872108560 2023-07-20 16:55:08,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-20 16:55:08,646 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x17c6cd47 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:55:08,649 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5b246c5e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:55:08,649 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-20 16:55:08,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-20 16:55:08,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:55:08,651 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:55:08,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,34355,1689872074166 to hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,34355,1689872074166-dead as it is dead 2023-07-20 16:55:08,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,34355,1689872074166-dead/jenkins-hbase4.apache.org%2C34355%2C1689872074166.jenkins-hbase4.apache.org%2C34355%2C1689872074166.regiongroup-0.1689872074292 2023-07-20 16:55:08,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,34355,1689872074166-dead/jenkins-hbase4.apache.org%2C34355%2C1689872074166.jenkins-hbase4.apache.org%2C34355%2C1689872074166.regiongroup-0.1689872074292 after 1ms 2023-07-20 16:55:08,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,34355,1689872074166-dead/jenkins-hbase4.apache.org%2C34355%2C1689872074166.jenkins-hbase4.apache.org%2C34355%2C1689872074166.regiongroup-0.1689872074292 to hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34355%2C1689872074166.jenkins-hbase4.apache.org%2C34355%2C1689872074166.regiongroup-0.1689872074292 2023-07-20 16:55:08,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,34355,1689872074166-dead 2023-07-20 16:55:08,659 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,39055,1689872108560 2023-07-20 16:55:08,660 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:55:08,661 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39055%2C1689872108560.jenkins-hbase4.apache.org%2C39055%2C1689872108560.regiongroup-0, suffix=, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,39055,1689872108560, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/oldWALs, maxLogs=10 2023-07-20 16:55:08,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,39055,1689872108560/jenkins-hbase4.apache.org%2C39055%2C1689872108560.jenkins-hbase4.apache.org%2C39055%2C1689872108560.regiongroup-0.1689872108662 2023-07-20 16:55:08,674 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK], DatanodeInfoWithStorage[127.0.0.1:35847,DS-e3134465-ba87-4999-ba03-bbcb207f3c3b,DISK], DatanodeInfoWithStorage[127.0.0.1:36101,DS-69ae4973-7589-41c7-bb37-0af0ca263f5e,DISK]] 2023-07-20 16:55:08,674 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:55:08,674 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:55:08,674 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:55:08,674 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:55:08,679 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:55:08,679 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-20 16:55:08,680 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-20 16:55:08,686 INFO [StoreFileOpener-proc-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b9a23bbf9b647e9b1922993c9037ba7 2023-07-20 16:55:08,686 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3b9a23bbf9b647e9b1922993c9037ba7 2023-07-20 16:55:08,690 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/56183e65ade64a5ab9377e2784bcf14d 2023-07-20 16:55:08,690 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:08,691 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-20 16:55:08,691 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34355%2C1689872074166.jenkins-hbase4.apache.org%2C34355%2C1689872074166.regiongroup-0.1689872074292 2023-07-20 16:55:08,695 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 140, firstSequenceIdInLog=120, maxSequenceIdInLog=241, path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34355%2C1689872074166.jenkins-hbase4.apache.org%2C34355%2C1689872074166.regiongroup-0.1689872074292 2023-07-20 16:55:08,695 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C34355%2C1689872074166.jenkins-hbase4.apache.org%2C34355%2C1689872074166.regiongroup-0.1689872074292 2023-07-20 16:55:08,698 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-20 16:55:08,700 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/241.seqid, newMaxSeqId=241, maxSeqId=118 2023-07-20 16:55:08,701 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=242; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74233796, jitterRate=0.10616976022720337}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 16:55:08,701 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 16:55:08,701 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-20 16:55:08,702 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-20 16:55:08,703 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-20 16:55:08,703 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-20 16:55:08,703 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-20 16:55:08,709 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-20 16:55:08,709 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-20 16:55:08,709 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-20 16:55:08,709 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-20 16:55:08,710 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=t1_syncup 2023-07-20 16:55:08,710 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=t2_syncup 2023-07-20 16:55:08,710 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-20 16:55:08,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-20 16:55:08,711 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-20 16:55:08,712 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46817,1689872074194, table=hbase:meta, region=1588230740 2023-07-20 16:55:08,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 1 possibly 'live' servers, and 0 'splitting'. 2023-07-20 16:55:08,714 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,46817,1689872074194 already deleted, retry=false 2023-07-20 16:55:08,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,46817,1689872074194 on jenkins-hbase4.apache.org,39055,1689872108560 2023-07-20 16:55:08,717 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,46817,1689872074194, splitWal=true, meta=true 2023-07-20 16:55:08,717 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=27 for jenkins-hbase4.apache.org,46817,1689872074194 (carryingMeta=true) jenkins-hbase4.apache.org,46817,1689872074194/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@6c5408d5[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-20 16:55:08,727 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-20 16:55:08,727 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-20 16:55:08,728 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/balancer 2023-07-20 16:55:08,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-20 16:55:08,728 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/normalizer 2023-07-20 16:55:08,729 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/split 2023-07-20 16:55:08,729 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/switch/merge 2023-07-20 16:55:08,730 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/snapshot-cleanup 2023-07-20 16:55:08,731 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 16:55:08,731 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:44889-0x10183a7c1310034, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/2/running 2023-07-20 16:55:08,731 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:55:08,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,39055,1689872108560, sessionid=0x10183a7c1310033, setting cluster-up flag (Was=false) 2023-07-20 16:55:08,737 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/flush-table-proc/acquired, /2/flush-table-proc/reached, /2/flush-table-proc/abort 2023-07-20 16:55:08,738 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39055,1689872108560 2023-07-20 16:55:08,741 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /2/online-snapshot/acquired, /2/online-snapshot/reached, /2/online-snapshot/abort 2023-07-20 16:55:08,742 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39055,1689872108560 2023-07-20 16:55:08,742 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/.hbase-snapshot/.tmp 2023-07-20 16:55:08,743 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:55:08,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:55:08,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:55:08,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-20 16:55:08,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-20 16:55:08,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:08,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 16:55:08,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:08,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689872138751 2023-07-20 16:55:08,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-20 16:55:08,752 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-20 16:55:08,752 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-20 16:55:08,752 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-20 16:55:08,752 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-20 16:55:08,752 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-20 16:55:08,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:08,755 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46817,1689872074194; numProcessing=1 2023-07-20 16:55:08,755 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=27, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46817,1689872074194, splitWal=true, meta=true 2023-07-20 16:55:08,756 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-20 16:55:08,756 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-20 16:55:08,756 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-20 16:55:08,757 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-20 16:55:08,757 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-20 16:55:08,757 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872108757,5,FailOnTimeoutGroup] 2023-07-20 16:55:08,757 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872108757,5,FailOnTimeoutGroup] 2023-07-20 16:55:08,757 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46817,1689872074194, splitWal=true, meta=true, isMeta: true 2023-07-20 16:55:08,757 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:08,758 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-20 16:55:08,758 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:08,758 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:08,758 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689872108758, completionTime=-1 2023-07-20 16:55:08,758 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-20 16:55:08,758 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-20 16:55:08,759 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194-splitting 2023-07-20 16:55:08,760 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194-splitting dir is empty, no logs to split. 2023-07-20 16:55:08,760 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46817,1689872074194 WAL count=0, meta=true 2023-07-20 16:55:08,763 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194-splitting dir is empty, no logs to split. 2023-07-20 16:55:08,763 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46817,1689872074194 WAL count=0, meta=true 2023-07-20 16:55:08,763 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46817,1689872074194 WAL splitting is done? wals=0, meta=true 2023-07-20 16:55:08,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-20 16:55:08,766 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-20 16:55:08,768 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 16:55:08,802 INFO [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(951): ClusterId : de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:55:08,802 DEBUG [RS:0;jenkins-hbase4:44889] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-20 16:55:08,804 DEBUG [RS:0;jenkins-hbase4:44889] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-20 16:55:08,804 DEBUG [RS:0;jenkins-hbase4:44889] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-20 16:55:08,806 DEBUG [RS:0;jenkins-hbase4:44889] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-20 16:55:08,808 DEBUG [RS:0;jenkins-hbase4:44889] zookeeper.ReadOnlyZKClient(139): Connect 0x39f4c5ed to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:55:08,811 DEBUG [RS:0;jenkins-hbase4:44889] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@37332ca3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:55:08,811 DEBUG [RS:0;jenkins-hbase4:44889] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@330cae9d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 16:55:08,820 DEBUG [RS:0;jenkins-hbase4:44889] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:44889 2023-07-20 16:55:08,820 INFO [RS:0;jenkins-hbase4:44889] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-20 16:55:08,820 INFO [RS:0;jenkins-hbase4:44889] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-20 16:55:08,820 DEBUG [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(1022): About to register with Master. 2023-07-20 16:55:08,820 INFO [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39055,1689872108560 with isa=jenkins-hbase4.apache.org/172.31.14.131:44889, startcode=1689872108583 2023-07-20 16:55:08,820 DEBUG [RS:0;jenkins-hbase4:44889] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-20 16:55:08,824 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52465, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=RegionServerStatusService 2023-07-20 16:55:08,824 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39055] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:08,825 DEBUG [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a 2023-07-20 16:55:08,825 DEBUG [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33767 2023-07-20 16:55:08,825 DEBUG [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-20 16:55:08,828 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310027, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:55:08,828 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:55:08,828 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 16:55:08,828 DEBUG [RS:0;jenkins-hbase4:44889] zookeeper.ZKUtil(162): regionserver:44889-0x10183a7c1310034, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:08,829 WARN [RS:0;jenkins-hbase4:44889] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-20 16:55:08,829 INFO [RS:0;jenkins-hbase4:44889] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:55:08,829 INFO [RS:0;jenkins-hbase4:44889] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:55:08,829 DEBUG [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:08,830 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,44889,1689872108583] 2023-07-20 16:55:08,835 DEBUG [RS:0;jenkins-hbase4:44889] zookeeper.ZKUtil(162): regionserver:44889-0x10183a7c1310034, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/rs/jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:08,836 DEBUG [RS:0;jenkins-hbase4:44889] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 16:55:08,836 INFO [RS:0;jenkins-hbase4:44889] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-20 16:55:08,838 INFO [RS:0;jenkins-hbase4:44889] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-20 16:55:08,838 INFO [RS:0;jenkins-hbase4:44889] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-20 16:55:08,838 INFO [RS:0;jenkins-hbase4:44889] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:08,840 INFO [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-20 16:55:08,842 INFO [RS:0;jenkins-hbase4:44889] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:08,843 DEBUG [RS:0;jenkins-hbase4:44889] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:08,844 DEBUG [RS:0;jenkins-hbase4:44889] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:08,844 DEBUG [RS:0;jenkins-hbase4:44889] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:08,844 DEBUG [RS:0;jenkins-hbase4:44889] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:08,844 DEBUG [RS:0;jenkins-hbase4:44889] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:08,844 DEBUG [RS:0;jenkins-hbase4:44889] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-20 16:55:08,844 DEBUG [RS:0;jenkins-hbase4:44889] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:08,844 DEBUG [RS:0;jenkins-hbase4:44889] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:08,844 DEBUG [RS:0;jenkins-hbase4:44889] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:08,844 DEBUG [RS:0;jenkins-hbase4:44889] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-20 16:55:08,845 INFO [RS:0;jenkins-hbase4:44889] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:08,845 INFO [RS:0;jenkins-hbase4:44889] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:08,845 INFO [RS:0;jenkins-hbase4:44889] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:08,857 INFO [RS:0;jenkins-hbase4:44889] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-20 16:55:08,857 INFO [RS:0;jenkins-hbase4:44889] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44889,1689872108583-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:08,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=101ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-20 16:55:08,867 INFO [RS:0;jenkins-hbase4:44889] regionserver.Replication(203): jenkins-hbase4.apache.org,44889,1689872108583 started 2023-07-20 16:55:08,867 INFO [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,44889,1689872108583, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:44889, sessionid=0x10183a7c1310034 2023-07-20 16:55:08,867 DEBUG [RS:0;jenkins-hbase4:44889] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-20 16:55:08,867 DEBUG [RS:0;jenkins-hbase4:44889] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:08,867 DEBUG [RS:0;jenkins-hbase4:44889] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44889,1689872108583' 2023-07-20 16:55:08,867 DEBUG [RS:0;jenkins-hbase4:44889] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/flush-table-proc/abort' 2023-07-20 16:55:08,868 DEBUG [RS:0;jenkins-hbase4:44889] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/flush-table-proc/acquired' 2023-07-20 16:55:08,868 DEBUG [RS:0;jenkins-hbase4:44889] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-20 16:55:08,868 DEBUG [RS:0;jenkins-hbase4:44889] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-20 16:55:08,868 DEBUG [RS:0;jenkins-hbase4:44889] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:08,868 DEBUG [RS:0;jenkins-hbase4:44889] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44889,1689872108583' 2023-07-20 16:55:08,868 DEBUG [RS:0;jenkins-hbase4:44889] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/2/online-snapshot/abort' 2023-07-20 16:55:08,869 DEBUG [RS:0;jenkins-hbase4:44889] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/2/online-snapshot/acquired' 2023-07-20 16:55:08,869 DEBUG [RS:0;jenkins-hbase4:44889] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-20 16:55:08,869 INFO [RS:0;jenkins-hbase4:44889] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-20 16:55:08,869 INFO [RS:0;jenkins-hbase4:44889] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-20 16:55:08,918 DEBUG [jenkins-hbase4:39055] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-20 16:55:08,919 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,44889,1689872108583, state=OPENING 2023-07-20 16:55:08,921 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-20 16:55:08,921 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-20 16:55:08,921 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,44889,1689872108583}] 2023-07-20 16:55:08,969 INFO [RS:0;jenkins-hbase4:44889] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:55:08,971 INFO [RS:0;jenkins-hbase4:44889] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44889%2C1689872108583.jenkins-hbase4.apache.org%2C44889%2C1689872108583.regiongroup-0, suffix=, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,44889,1689872108583, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs, maxLogs=10 2023-07-20 16:55:08,983 INFO [RS:0;jenkins-hbase4:44889] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,44889,1689872108583/jenkins-hbase4.apache.org%2C44889%2C1689872108583.jenkins-hbase4.apache.org%2C44889%2C1689872108583.regiongroup-0.1689872108971 2023-07-20 16:55:08,983 DEBUG [RS:0;jenkins-hbase4:44889] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35885,DS-21386278-ed61-414b-9d04-87c474a58f9b,DISK], DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK], DatanodeInfoWithStorage[127.0.0.1:36101,DS-69ae4973-7589-41c7-bb37-0af0ca263f5e,DISK]] 2023-07-20 16:55:09,077 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:09,077 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 16:55:09,080 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46192, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 16:55:09,083 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-20 16:55:09,083 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:55:09,083 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:55:09,083 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:55:09,085 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44889%2C1689872108583.meta, suffix=.meta, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,44889,1689872108583, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs, maxLogs=10 2023-07-20 16:55:09,092 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,44889,1689872108583/jenkins-hbase4.apache.org%2C44889%2C1689872108583.meta.1689872109085.meta 2023-07-20 16:55:09,092 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35885,DS-21386278-ed61-414b-9d04-87c474a58f9b,DISK], DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK], DatanodeInfoWithStorage[127.0.0.1:36101,DS-69ae4973-7589-41c7-bb37-0af0ca263f5e,DISK]] 2023-07-20 16:55:09,092 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:55:09,093 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-20 16:55:09,093 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-20 16:55:09,093 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-20 16:55:09,093 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-20 16:55:09,093 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:55:09,093 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-20 16:55:09,093 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-20 16:55:09,094 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-20 16:55:09,095 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info 2023-07-20 16:55:09,095 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info 2023-07-20 16:55:09,095 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-20 16:55:09,102 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info/1bbd25e4897c48f2a459de39ee118f1a 2023-07-20 16:55:09,107 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info/730b9259ce88426992f0c3d8aad85399 2023-07-20 16:55:09,107 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:09,107 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-20 16:55:09,109 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/rep_barrier 2023-07-20 16:55:09,109 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/rep_barrier 2023-07-20 16:55:09,109 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-20 16:55:09,110 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:09,110 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-20 16:55:09,111 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/table 2023-07-20 16:55:09,111 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/table 2023-07-20 16:55:09,111 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-20 16:55:09,117 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/table/1619f7b97a8e4850aae967c3b3d7b384 2023-07-20 16:55:09,117 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:09,118 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740 2023-07-20 16:55:09,119 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740 2023-07-20 16:55:09,122 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-20 16:55:09,123 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-20 16:55:09,124 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=35; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65520352, jitterRate=-0.023670673370361328}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-20 16:55:09,124 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-20 16:55:09,127 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=29, masterSystemTime=1689872109077 2023-07-20 16:55:09,132 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-20 16:55:09,132 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-20 16:55:09,133 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,44889,1689872108583, state=OPEN 2023-07-20 16:55:09,134 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/meta-region-server 2023-07-20 16:55:09,134 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /2/meta-region-server: CHANGED 2023-07-20 16:55:09,137 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-07-20 16:55:09,137 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,44889,1689872108583 in 213 msec 2023-07-20 16:55:09,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-07-20 16:55:09,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 373 msec 2023-07-20 16:55:10,361 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=1603ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-07-20 16:55:11,574 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 16:55:11,864 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=3106ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-20 16:55:13,266 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=4508ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-20 16:55:13,266 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-20 16:55:13,270 DEBUG [hconnection-0x2b4f992b-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:55:13,272 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35990, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:55:13,280 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=7ebeef24b783c4189ed905a5659b7e20, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46817,1689872074194, regionLocation=jenkins-hbase4.apache.org,46817,1689872074194, openSeqNum=10 2023-07-20 16:55:13,280 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=058a0bfd9d8bf39d8c91089a151cb277, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46817,1689872074194, regionLocation=jenkins-hbase4.apache.org,46817,1689872074194, openSeqNum=11 2023-07-20 16:55:13,281 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=c96c52d963776c1249f8d28dc1cee95e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46817,1689872074194, regionLocation=jenkins-hbase4.apache.org,46817,1689872074194, openSeqNum=12 2023-07-20 16:55:13,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-20 16:55:13,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689872173281 2023-07-20 16:55:13,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689872233281 2023-07-20 16:55:13,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 14 msec 2023-07-20 16:55:13,304 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,46817,1689872074194 had 4 regions 2023-07-20 16:55:13,304 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39055,1689872108560-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:13,304 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39055,1689872108560-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:13,304 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39055,1689872108560-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:13,304 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:39055, period=300000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:13,304 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-20 16:55:13,305 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. is NOT online; state={7ebeef24b783c4189ed905a5659b7e20 state=OPEN, ts=1689872113280, server=jenkins-hbase4.apache.org,46817,1689872074194}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-20 16:55:13,306 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=27, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46817,1689872074194, splitWal=true, meta=true, isMeta: false 2023-07-20 16:55:13,309 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,46817,1689872074194/hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20., unknown_server=jenkins-hbase4.apache.org,46817,1689872074194/t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277., unknown_server=jenkins-hbase4.apache.org,46817,1689872074194/t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:55:13,309 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194-splitting dir is empty, no logs to split. 2023-07-20 16:55:13,309 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,46817,1689872074194 WAL count=0, meta=false 2023-07-20 16:55:13,312 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,46817,1689872074194-splitting dir is empty, no logs to split. 2023-07-20 16:55:13,312 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,46817,1689872074194 WAL count=0, meta=false 2023-07-20 16:55:13,312 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46817,1689872074194 WAL splitting is done? wals=0, meta=false 2023-07-20 16:55:13,316 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=c96c52d963776c1249f8d28dc1cee95e, ASSIGN}, {pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=058a0bfd9d8bf39d8c91089a151cb277, ASSIGN}, {pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=7ebeef24b783c4189ed905a5659b7e20, ASSIGN}] 2023-07-20 16:55:13,317 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=7ebeef24b783c4189ed905a5659b7e20, ASSIGN 2023-07-20 16:55:13,317 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t2_syncup, region=c96c52d963776c1249f8d28dc1cee95e, ASSIGN 2023-07-20 16:55:13,318 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=t1_syncup, region=058a0bfd9d8bf39d8c91089a151cb277, ASSIGN 2023-07-20 16:55:13,318 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=7ebeef24b783c4189ed905a5659b7e20, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 16:55:13,319 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t2_syncup, region=c96c52d963776c1249f8d28dc1cee95e, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 16:55:13,319 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=t1_syncup, region=058a0bfd9d8bf39d8c91089a151cb277, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-20 16:55:13,469 DEBUG [jenkins-hbase4:39055] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-20 16:55:13,473 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=c96c52d963776c1249f8d28dc1cee95e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:13,473 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=058a0bfd9d8bf39d8c91089a151cb277, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:13,473 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=7ebeef24b783c4189ed905a5659b7e20, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:13,473 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872113473"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872113473"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872113473"}]},"ts":"1689872113473"} 2023-07-20 16:55:13,473 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872113473"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872113473"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872113473"}]},"ts":"1689872113473"} 2023-07-20 16:55:13,473 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689872113473"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689872113473"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689872113473"}]},"ts":"1689872113473"} 2023-07-20 16:55:13,476 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 7ebeef24b783c4189ed905a5659b7e20, server=jenkins-hbase4.apache.org,44889,1689872108583}] 2023-07-20 16:55:13,477 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure 058a0bfd9d8bf39d8c91089a151cb277, server=jenkins-hbase4.apache.org,44889,1689872108583}] 2023-07-20 16:55:13,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=35, ppid=30, state=RUNNABLE; OpenRegionProcedure c96c52d963776c1249f8d28dc1cee95e, server=jenkins-hbase4.apache.org,44889,1689872108583}] 2023-07-20 16:55:13,635 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:55:13,635 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.FSHLogProvider 2023-07-20 16:55:13,636 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44889%2C1689872108583.jenkins-hbase4.apache.org%2C44889%2C1689872108583.regiongroup-1, suffix=, logDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,44889,1689872108583, archiveDir=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs, maxLogs=10 2023-07-20 16:55:13,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:55:13,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 058a0bfd9d8bf39d8c91089a151cb277, NAME => 't1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:55:13,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t1_syncup 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:55:13,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:55:13,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:55:13,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:55:13,641 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:55:13,642 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1 2023-07-20 16:55:13,642 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1 2023-07-20 16:55:13,642 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 058a0bfd9d8bf39d8c91089a151cb277 columnFamilyName cf1 2023-07-20 16:55:13,652 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,44889,1689872108583/jenkins-hbase4.apache.org%2C44889%2C1689872108583.jenkins-hbase4.apache.org%2C44889%2C1689872108583.regiongroup-1.1689872113637 2023-07-20 16:55:13,652 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new FSHLog writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35885,DS-21386278-ed61-414b-9d04-87c474a58f9b,DISK], DatanodeInfoWithStorage[127.0.0.1:38315,DS-f901468c-90d8-4877-8572-70b06cbb96ab,DISK], DatanodeInfoWithStorage[127.0.0.1:35847,DS-e3134465-ba87-4999-ba03-bbcb207f3c3b,DISK]] 2023-07-20 16:55:13,652 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7ebeef24b783c4189ed905a5659b7e20, NAME => 'hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:55:13,652 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:55:13,652 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:55:13,653 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:55:13,653 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:55:13,654 INFO [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:55:13,655 DEBUG [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/info 2023-07-20 16:55:13,655 DEBUG [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/info 2023-07-20 16:55:13,655 INFO [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7ebeef24b783c4189ed905a5659b7e20 columnFamilyName info 2023-07-20 16:55:13,655 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1/806acc8751334bdeaac05f420a6fcdb3 2023-07-20 16:55:13,663 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb4f45b9e6a94e068dda624387ebacad 2023-07-20 16:55:13,663 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1/cb4f45b9e6a94e068dda624387ebacad 2023-07-20 16:55:13,663 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(310): Store=058a0bfd9d8bf39d8c91089a151cb277/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:13,664 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:55:13,665 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/norep 2023-07-20 16:55:13,665 DEBUG [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/norep 2023-07-20 16:55:13,665 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 058a0bfd9d8bf39d8c91089a151cb277 columnFamilyName norep 2023-07-20 16:55:13,666 INFO [StoreOpener-058a0bfd9d8bf39d8c91089a151cb277-1] regionserver.HStore(310): Store=058a0bfd9d8bf39d8c91089a151cb277/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:13,679 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:55:13,680 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:55:13,681 DEBUG [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/info/ab70d834ddd448fe9bb282fb38cf6ad4 2023-07-20 16:55:13,681 INFO [StoreOpener-7ebeef24b783c4189ed905a5659b7e20-1] regionserver.HStore(310): Store=7ebeef24b783c4189ed905a5659b7e20/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:13,682 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:55:13,683 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:55:13,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t1_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 16:55:13,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:55:13,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 058a0bfd9d8bf39d8c91089a151cb277; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71836036, jitterRate=0.07044035196304321}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 16:55:13,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 058a0bfd9d8bf39d8c91089a151cb277: 2023-07-20 16:55:13,689 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:55:13,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277., pid=34, masterSystemTime=1689872113631 2023-07-20 16:55:13,691 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7ebeef24b783c4189ed905a5659b7e20; next sequenceid=13; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62650328, jitterRate=-0.06643736362457275}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-20 16:55:13,691 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7ebeef24b783c4189ed905a5659b7e20: 2023-07-20 16:55:13,693 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20., pid=33, masterSystemTime=1689872113631 2023-07-20 16:55:13,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:55:13,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:55:13,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:55:13,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c96c52d963776c1249f8d28dc1cee95e, NAME => 't2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.', STARTKEY => '', ENDKEY => ''} 2023-07-20 16:55:13,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table t2_syncup c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:55:13,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-20 16:55:13,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:55:13,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:55:13,694 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=058a0bfd9d8bf39d8c91089a151cb277, regionState=OPEN, openSeqNum=18, regionLocation=jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:13,694 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872113694"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872113694"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872113694"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872113694"}]},"ts":"1689872113694"} 2023-07-20 16:55:13,695 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:55:13,695 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:55:13,696 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=7ebeef24b783c4189ed905a5659b7e20, regionState=OPEN, openSeqNum=13, regionLocation=jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:13,696 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689872113695"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872113695"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872113695"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872113695"}]},"ts":"1689872113695"} 2023-07-20 16:55:13,696 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:55:13,697 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1 2023-07-20 16:55:13,697 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1 2023-07-20 16:55:13,698 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c96c52d963776c1249f8d28dc1cee95e columnFamilyName cf1 2023-07-20 16:55:13,698 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-07-20 16:55:13,698 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure 058a0bfd9d8bf39d8c91089a151cb277, server=jenkins-hbase4.apache.org,44889,1689872108583 in 219 msec 2023-07-20 16:55:13,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-07-20 16:55:13,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 7ebeef24b783c4189ed905a5659b7e20, server=jenkins-hbase4.apache.org,44889,1689872108583 in 221 msec 2023-07-20 16:55:13,700 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t1_syncup, region=058a0bfd9d8bf39d8c91089a151cb277, ASSIGN in 384 msec 2023-07-20 16:55:13,701 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=7ebeef24b783c4189ed905a5659b7e20, ASSIGN in 386 msec 2023-07-20 16:55:13,709 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 34a7703742324ea7806222645495771e 2023-07-20 16:55:13,710 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1/34a7703742324ea7806222645495771e 2023-07-20 16:55:13,717 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(539): loaded hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1/6419e14a3bbe44399930ef7d7a7c0e8b 2023-07-20 16:55:13,717 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(310): Store=c96c52d963776c1249f8d28dc1cee95e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:13,717 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family norep of region c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:55:13,718 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/norep 2023-07-20 16:55:13,718 DEBUG [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/norep 2023-07-20 16:55:13,719 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c96c52d963776c1249f8d28dc1cee95e columnFamilyName norep 2023-07-20 16:55:13,719 INFO [StoreOpener-c96c52d963776c1249f8d28dc1cee95e-1] regionserver.HStore(310): Store=c96c52d963776c1249f8d28dc1cee95e/norep, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-20 16:55:13,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:55:13,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:55:13,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table t2_syncup descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-20 16:55:13,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:55:13,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c96c52d963776c1249f8d28dc1cee95e; next sequenceid=19; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68028234, jitterRate=0.01369968056678772}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-20 16:55:13,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c96c52d963776c1249f8d28dc1cee95e: 2023-07-20 16:55:13,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e., pid=35, masterSystemTime=1689872113631 2023-07-20 16:55:13,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:55:13,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:55:13,728 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=c96c52d963776c1249f8d28dc1cee95e, regionState=OPEN, openSeqNum=19, regionLocation=jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:13,728 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1689872113728"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689872113728"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689872113728"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689872113728"}]},"ts":"1689872113728"} 2023-07-20 16:55:13,731 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=35, resume processing ppid=30 2023-07-20 16:55:13,732 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=30, state=SUCCESS; OpenRegionProcedure c96c52d963776c1249f8d28dc1cee95e, server=jenkins-hbase4.apache.org,44889,1689872108583 in 252 msec 2023-07-20 16:55:13,735 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-07-20 16:55:13,735 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=t2_syncup, region=c96c52d963776c1249f8d28dc1cee95e, ASSIGN in 418 msec 2023-07-20 16:55:13,735 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,46817,1689872074194 after splitting done 2023-07-20 16:55:13,735 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,46817,1689872074194 from processing; numProcessing=0 2023-07-20 16:55:13,737 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,46817,1689872074194, splitWal=true, meta=true in 5.0210 sec 2023-07-20 16:55:13,867 INFO [jenkins-hbase4:44889Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:55:14,306 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on existing znode=/2/namespace 2023-07-20 16:55:14,319 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/default 2023-07-20 16:55:14,322 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/2/namespace/hbase 2023-07-20 16:55:14,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.711sec 2023-07-20 16:55:14,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-20 16:55:14,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-20 16:55:14,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-20 16:55:14,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39055,1689872108560-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-20 16:55:14,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39055,1689872108560-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-20 16:55:14,326 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-20 16:55:14,406 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(139): Connect 0x0efdbffd to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:55:14,411 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@72fb89fc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:55:14,412 DEBUG [hconnection-0x566ca1cd-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:55:14,414 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35996, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:55:14,416 INFO [Listener at localhost/33489] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-20 16:55:14,417 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0efdbffd to 127.0.0.1:56032 2023-07-20 16:55:14,417 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:14,418 INFO [Listener at localhost/33489] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:39055 after: jenkins-hbase4.apache.org:39055 2023-07-20 16:55:14,418 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(139): Connect 0x015a026f to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:55:14,429 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@320ba940, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:55:14,836 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't2_syncup' 2023-07-20 16:55:14,837 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-20 16:55:14,837 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-20 16:55:14,838 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 't1_syncup' 2023-07-20 16:55:14,931 DEBUG [hconnection-0x72db595b-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:55:14,934 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36004, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-20 16:55:14,954 INFO [Listener at localhost/33489] zookeeper.RecoverableZooKeeper(93): Process identifier=syncupReplication1689872114954 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:55:14,957 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): syncupReplication16898721149540x0, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:55:14,958 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): syncupReplication1689872114954-0x10183a7c1310039 connected Start Replication Server start 2023-07-20 16:55:14,960 INFO [Listener at localhost/33489] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.RegionGroupingProvider 2023-07-20 16:55:14,960 INFO [Listener at localhost/33489] wal.RegionGroupingProvider(103): Instantiating RegionGroupingStrategy of type class org.apache.hadoop.hbase.wal.BoundedGroupingStrategy 2023-07-20 16:55:14,972 INFO [Listener at localhost/33489] hbase.HBaseConfiguration(293): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-07-20 16:55:14,973 DEBUG [Listener at localhost/33489] regionserver.Replication(139): Replication stats-in-log period=5 seconds 2023-07-20 16:55:14,983 INFO [Listener at localhost/33489] regionserver.ReplicationSource(232): queueId=1, ReplicationSource: 1, currentBandwidth=0 2023-07-20 16:55:14,984 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager(266): Current list of replicators: [jenkins-hbase4.apache.org,41127,1689872100966] other RSs: [] 2023-07-20 16:55:15,003 DEBUG [Listener at localhost/33489.replicationSource,1] zookeeper.ReadOnlyZKClient(139): Connect 0x07bd1b44 to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:55:15,007 DEBUG [Listener at localhost/33489.replicationSource,1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@7d0c6648, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:55:15,007 INFO [Listener at localhost/33489.replicationSource,1] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:55:15,012 DEBUG [Listener at localhost/33489.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:55:15,014 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10183a7c131003b connected 2023-07-20 16:55:15,014 INFO [Listener at localhost/33489.replicationSource,1] regionserver.ReplicationSource(586): peerId=1, queueId=1 (queues=0) is replicating from cluster=258804e0-584b-4fe6-b46a-945d0467644e to cluster=de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:55:17,645 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(390): Atomically moving jenkins-hbase4.apache.org,41127,1689872100966/1's WALs to 1689872114960.syncuptool.replication.org,1234,1 2023-07-20 16:55:17,647 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0.1689872101395 with data PBUF\x08\xD8\x05 2023-07-20 16:55:17,647 DEBUG [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(415): Creating jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1.1689872106067 with data PBUF\x08\xB7\x06 2023-07-20 16:55:17,649 INFO [ReplicationExecutor-0] replication.ZKReplicationQueueStorage(433): Atomically moved jenkins-hbase4.apache.org,41127,1689872100966/1's WALs to 1689872114960.syncuptool.replication.org,1234,1 2023-07-20 16:55:18,867 INFO [jenkins-hbase4:44889Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:55:19,142 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 16:55:19,650 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,41127,1689872100966] 2023-07-20 16:55:19,651 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,41127,1689872100966] 2023-07-20 16:55:19,661 DEBUG [ReplicationExecutor-0] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,41127,1689872100966] 2023-07-20 16:55:19,661 INFO [ReplicationExecutor-0] regionserver.ReplicationSource(232): queueId=1-jenkins-hbase4.apache.org,41127,1689872100966, ReplicationSource: 1, currentBandwidth=0 2023-07-20 16:55:19,662 INFO [ReplicationExecutor-0] regionserver.ReplicationSourceManager$NodeFailoverWorker(800): Added recovered source 1-jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:19,682 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966] zookeeper.ReadOnlyZKClient(139): Connect 0x4452d20a to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:55:19,686 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@78f9d8df, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:55:19,686 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966] zookeeper.RecoverableZooKeeper(93): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:56032 2023-07-20 16:55:19,688 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 10x0, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-20 16:55:19,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): connection to cluster: 1-0x10183a7c131003d connected 2023-07-20 16:55:19,689 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSource(586): peerId=1, queueId=1-jenkins-hbase4.apache.org,41127,1689872100966 (queues=2) is replicating from cluster=258804e0-584b-4fe6-b46a-945d0467644e to cluster=de676223-472d-4f01-9b7c-7dc5661e1101 2023-07-20 16:55:19,690 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0 2023-07-20 16:55:19,693 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,41127,1689872100966, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 16:55:19,697 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSource(346): peerId=1, starting shipping worker for walGroupId=jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1 2023-07-20 16:55:19,697 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0 2023-07-20 16:55:19,698 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSourceWALReader(119): peerClusterZnode=1-jenkins-hbase4.apache.org,41127,1689872100966, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-07-20 16:55:19,700 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1 2023-07-20 16:55:19,703 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0.1689872101395 2023-07-20 16:55:19,703 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-20 16:55:19,703 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,41127,1689872100966 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 16:55:19,703 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0 of peer 1-jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:19,709 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.WALEntryStream(248): EOF, closing hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1.1689872106067 2023-07-20 16:55:19,709 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSourceWALReader(162): Read 300 WAL entries eligible for replication 2023-07-20 16:55:19,709 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.wal-reader.jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSourceWALReader(247): Stopping the replication source wal reader 2023-07-20 16:55:19,711 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,41127,1689872100966 got entry batch from reader: WALEntryBatch [walEntries=[{t1_syncup/ea29850f58e32db8bff448bce6d2160a/112=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/113=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/114=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/115=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/116=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/117=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/118=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/119=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/120=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/121=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/122=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/123=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/124=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/125=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/126=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/127=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/128=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/129=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/130=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/131=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/132=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/133=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/134=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/135=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/136=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/137=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/138=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/139=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/140=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/141=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/142=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/143=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/144=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/145=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/146=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/147=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/148=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/149=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/150=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/151=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/152=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/153=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/154=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/155=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/156=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/157=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/158=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/159=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/160=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/161=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/162=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/163=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/164=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/165=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/166=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/167=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/168=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/169=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/170=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/171=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/172=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/173=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/174=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/175=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/176=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/177=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/178=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/179=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/180=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/181=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/182=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/183=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/184=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/185=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/186=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/187=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/188=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/189=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/190=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/191=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/192=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/193=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/194=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/195=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/196=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/197=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/198=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/199=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/200=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/201=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/202=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/203=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/204=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/205=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/206=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/207=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/208=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/209=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/210=[#edits: 1 = ],208}, {t1_syncup/ea29850f58e32db8bff448bce6d2160a/211=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/212=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/213=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/214=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/215=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/216=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/217=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/218=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/219=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/220=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/221=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/222=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/223=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/224=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/225=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/226=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/227=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/228=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/229=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/230=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/231=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/232=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/233=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/234=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/235=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/236=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/237=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/238=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/239=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/240=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/241=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/242=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/243=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/244=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/245=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/246=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/247=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/248=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/249=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/250=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/251=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/252=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/253=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/254=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/255=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/256=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/257=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/258=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/259=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/260=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/261=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/262=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/263=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/264=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/265=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/266=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/267=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/268=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/269=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/270=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/271=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/272=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/273=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/274=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/275=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/276=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/277=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/278=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/279=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/280=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/281=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/282=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/283=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/284=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/285=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/286=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/287=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/288=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/289=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/290=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/291=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/292=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/293=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/294=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/295=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/296=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/297=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/298=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/299=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/300=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/301=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/302=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/303=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/304=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/305=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/306=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/307=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/308=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/309=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/310=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/311=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/312=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/313=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/314=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/315=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/316=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/317=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/318=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/319=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/320=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/321=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/322=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/323=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/324=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/325=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/326=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/327=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/328=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/329=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/330=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/331=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/332=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/333=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/334=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/335=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/336=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/337=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/338=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/339=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/340=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/341=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/342=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/343=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/344=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/345=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/346=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/347=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/348=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/349=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/350=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/351=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/352=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/353=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/354=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/355=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/356=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/357=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/358=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/359=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/360=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/361=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/362=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/363=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/364=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/365=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/366=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/367=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/368=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/369=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/370=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/371=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/372=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/373=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/374=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/375=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/376=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/377=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/378=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/379=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/380=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/381=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/382=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/383=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/384=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/385=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/386=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/387=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/388=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/389=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/390=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/391=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/392=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/393=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/394=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/395=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/396=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/397=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/398=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/399=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/400=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/401=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/402=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/403=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/404=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/405=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/406=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/407=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/408=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/409=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/410=[#edits: 1 = ],208}, {t2_syncup/2627e10c9cf6db0694d7b55799cf6705/411=[#edits: 1 = ],208}], lastWalPath=hdfs://localhost:38055/user/jenkins/test-data/7002b0ec-868a-cb73-c924-d99c087dd738/oldWALs/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1.1689872106067, lastWalPosition=34707, nbRowKeys=300, nbHFiles=0, heapSize=62400, lastSeqIds={}, endOfFile=true,usedBufferSize=62400] 2023-07-20 16:55:19,720 DEBUG [SinkThread-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-20 16:55:19,731 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36010, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-20 16:55:19,735 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44889] regionserver.ReplicationSink(263): Started replicating mutations. 2023-07-20 16:55:19,736 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44889] zookeeper.ReadOnlyZKClient(139): Connect 0x4acd0e1c to 127.0.0.1:56032 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-07-20 16:55:19,741 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44889] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@440ae935, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-20 16:55:19,743 DEBUG [hconnection-0x4035656c-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-20 16:55:19,746 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36024, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=ClientService 2023-07-20 16:55:19,770 DEBUG [RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44889] regionserver.ReplicationSink(267): Finished replicating mutations. 2023-07-20 16:55:19,771 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1-jenkins-hbase4.apache.org,41127,1689872100966] replication.ReplicationQueueInfo(107): Found dead servers:[jenkins-hbase4.apache.org,41127,1689872100966] 2023-07-20 16:55:19,773 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSourceManager(580): Removing 1 logs in the list: [jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1.1689872106067] 2023-07-20 16:55:19,775 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSourceShipper(110): Shipper from source 1-jenkins-hbase4.apache.org,41127,1689872100966 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-07-20 16:55:19,775 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.RecoveredReplicationSourceShipper(50): Finished recovering queue for group jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1 of peer 1-jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:19,804 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:19,805 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1-jenkins-hbase4.apache.org,41127,1689872100966] zookeeper.ZKUtil(162): syncupReplication1689872114954-0x10183a7c1310039, quorum=127.0.0.1:56032, baseZNode=/1 Set watcher on existing znode=/1/replication/rs/1689872114960.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,41127,1689872100966/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0.1689872101395 2023-07-20 16:55:19,806 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689872114954-0x10183a7c1310039, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689872114960.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,41127,1689872100966/jenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0.1689872101395 2023-07-20 16:55:19,806 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689872114954-0x10183a7c1310039, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/1/replication/rs/1689872114960.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:19,806 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-0,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,41127,1689872100966 with the following stats: Total replicated edits: 300, current progress: 2023-07-20 16:55:19,806 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689872114954-0x10183a7c1310039, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/1/replication/rs/1689872114960.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:19,907 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.ReplicationSourceManager(453): Done with the recovered queue 1-jenkins-hbase4.apache.org,41127,1689872100966 2023-07-20 16:55:19,907 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1-jenkins-hbase4.apache.org,41127,1689872100966] zookeeper.ZKUtil(243): syncupReplication1689872114954-0x10183a7c1310039, quorum=127.0.0.1:56032, baseZNode=/1 Unable to list children of znode /1/replication/rs/1689872114960.syncuptool.replication.org,1234,1/1-jenkins-hbase4.apache.org,41127,1689872100966 because node does not exist (not an error) 2023-07-20 16:55:19,908 INFO [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966.replicationSource.shipperjenkins-hbase4.apache.org%2C41127%2C1689872100966.jenkins-hbase4.apache.org%2C41127%2C1689872100966.regiongroup-1,1-jenkins-hbase4.apache.org,41127,1689872100966] regionserver.RecoveredReplicationSource(153): Finished recovering queue 1-jenkins-hbase4.apache.org,41127,1689872100966 with the following stats: Total replicated edits: 300, current progress: 2023-07-20 16:55:23,867 INFO [jenkins-hbase4:44889Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-07-20 16:55:23,867 INFO [jenkins-hbase4:44889Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Sink: age in ms of last applied edit: 0, total replicated edits: 300 2023-07-20 16:55:24,984 INFO [Listener at localhost/33489] regionserver.ReplicationSource(656): peerId=1, Closing source 1 because: Region server is closing 2023-07-20 16:55:25,091 DEBUG [Listener at localhost/33489.replicationSource,1-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c131003b, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:25,091 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x07bd1b44 to 127.0.0.1:56032 2023-07-20 16:55:25,092 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:25,294 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689872114954-0x10183a7c1310039, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:25,294 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): syncupReplication1689872114954-0x10183a7c1310039, quorum=127.0.0.1:56032, baseZNode=/1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:25,302 INFO [Listener at localhost/33489] replication.TestReplicationSyncUpTool(280): SyncUpAfterPut succeeded at retry = 0 2023-07-20 16:55:25,302 INFO [Listener at localhost/33489] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-20 16:55:25,302 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x015a026f to 127.0.0.1:56032 2023-07-20 16:55:25,303 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:25,303 DEBUG [Listener at localhost/33489] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-20 16:55:25,303 DEBUG [Listener at localhost/33489] util.JVMClusterUtil(257): Found active master hash=424197051, stopped=false 2023-07-20 16:55:25,303 INFO [Listener at localhost/33489] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,39055,1689872108560 2023-07-20 16:55:25,304 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 16:55:25,304 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:44889-0x10183a7c1310034, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/running 2023-07-20 16:55:25,305 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:55:25,304 INFO [Listener at localhost/33489] procedure2.ProcedureExecutor(629): Stopping 2023-07-20 16:55:25,305 DEBUG [Listener at localhost/33489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x17c6cd47 to 127.0.0.1:56032 2023-07-20 16:55:25,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44889-0x10183a7c1310034, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:55:25,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/running 2023-07-20 16:55:25,308 DEBUG [Listener at localhost/33489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:25,309 INFO [Listener at localhost/33489] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,44889,1689872108583' ***** 2023-07-20 16:55:25,309 INFO [Listener at localhost/33489] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-20 16:55:25,309 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-07-20 16:55:25,309 INFO [RS:0;jenkins-hbase4:44889] regionserver.HeapMemoryManager(220): Stopping 2023-07-20 16:55:25,311 INFO [RS:0;jenkins-hbase4:44889] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-20 16:55:25,311 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-20 16:55:25,311 INFO [RS:0;jenkins-hbase4:44889] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-20 16:55:25,312 INFO [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(3305): Received CLOSE for 7ebeef24b783c4189ed905a5659b7e20 2023-07-20 16:55:25,312 INFO [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(3305): Received CLOSE for c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:55:25,312 INFO [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(3305): Received CLOSE for 058a0bfd9d8bf39d8c91089a151cb277 2023-07-20 16:55:25,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7ebeef24b783c4189ed905a5659b7e20, disabling compactions & flushes 2023-07-20 16:55:25,313 INFO [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:25,313 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:55:25,313 DEBUG [RS:0;jenkins-hbase4:44889] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x39f4c5ed to 127.0.0.1:56032 2023-07-20 16:55:25,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:55:25,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. after waiting 0 ms 2023-07-20 16:55:25,313 DEBUG [RS:0;jenkins-hbase4:44889] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:25,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:55:25,313 INFO [RS:0;jenkins-hbase4:44889] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-20 16:55:25,313 INFO [RS:0;jenkins-hbase4:44889] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-20 16:55:25,313 INFO [RS:0;jenkins-hbase4:44889] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-20 16:55:25,313 INFO [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-20 16:55:25,319 INFO [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-20 16:55:25,319 DEBUG [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(1478): Online Regions={7ebeef24b783c4189ed905a5659b7e20=hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20., c96c52d963776c1249f8d28dc1cee95e=t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e., 1588230740=hbase:meta,,1.1588230740, 058a0bfd9d8bf39d8c91089a151cb277=t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277.} 2023-07-20 16:55:25,319 DEBUG [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(1504): Waiting on 058a0bfd9d8bf39d8c91089a151cb277, 1588230740, 7ebeef24b783c4189ed905a5659b7e20, c96c52d963776c1249f8d28dc1cee95e 2023-07-20 16:55:25,332 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-20 16:55:25,332 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-20 16:55:25,332 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-20 16:55:25,332 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-20 16:55:25,332 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-20 16:55:25,332 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.72 KB heapSize=5.24 KB 2023-07-20 16:55:25,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/namespace/7ebeef24b783c4189ed905a5659b7e20/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=12 2023-07-20 16:55:25,341 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:55:25,341 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7ebeef24b783c4189ed905a5659b7e20: 2023-07-20 16:55:25,341 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689872066061.7ebeef24b783c4189ed905a5659b7e20. 2023-07-20 16:55:25,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c96c52d963776c1249f8d28dc1cee95e, disabling compactions & flushes 2023-07-20 16:55:25,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:55:25,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:55:25,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. after waiting 0 ms 2023-07-20 16:55:25,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:55:25,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing c96c52d963776c1249f8d28dc1cee95e 2/2 column families, dataSize=7.01 KB heapSize=22.38 KB 2023-07-20 16:55:25,362 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:55:25,378 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.72 KB at sequenceid=43 (bloomFilter=false), to=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/.tmp/info/5bc3e27029684299bee6d410aa906015 2023-07-20 16:55:25,390 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.01 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/.tmp/cf1/e31ed68dc196436d84963fb577e4fda7 2023-07-20 16:55:25,397 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/.tmp/info/5bc3e27029684299bee6d410aa906015 as hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info/5bc3e27029684299bee6d410aa906015 2023-07-20 16:55:25,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/.tmp/cf1/e31ed68dc196436d84963fb577e4fda7 as hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1/e31ed68dc196436d84963fb577e4fda7 2023-07-20 16:55:25,410 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/cf1/e31ed68dc196436d84963fb577e4fda7, entries=200, sequenceid=22, filesize=12.2 K 2023-07-20 16:55:25,411 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~7.01 KB/7180, heapSize ~22.11 KB/22640, currentSize=0 B/0 for c96c52d963776c1249f8d28dc1cee95e in 69ms, sequenceid=22, compaction requested=true 2023-07-20 16:55:25,416 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/info/5bc3e27029684299bee6d410aa906015, entries=24, sequenceid=43, filesize=7.4 K 2023-07-20 16:55:25,429 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.72 KB/2786, heapSize ~4.73 KB/4840, currentSize=0 B/0 for 1588230740 in 97ms, sequenceid=43, compaction requested=true 2023-07-20 16:55:25,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t2_syncup/c96c52d963776c1249f8d28dc1cee95e/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=18 2023-07-20 16:55:25,452 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:55:25,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c96c52d963776c1249f8d28dc1cee95e: 2023-07-20 16:55:25,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t2_syncup,,1689872068593.c96c52d963776c1249f8d28dc1cee95e. 2023-07-20 16:55:25,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 058a0bfd9d8bf39d8c91089a151cb277, disabling compactions & flushes 2023-07-20 16:55:25,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:55:25,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:55:25,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. after waiting 0 ms 2023-07-20 16:55:25,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:55:25,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 058a0bfd9d8bf39d8c91089a151cb277 2/2 column families, dataSize=3.40 KB heapSize=11.44 KB 2023-07-20 16:55:25,455 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/hbase/meta/1588230740/recovered.edits/46.seqid, newMaxSeqId=46, maxSeqId=34 2023-07-20 16:55:25,455 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-20 16:55:25,456 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-20 16:55:25,456 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-20 16:55:25,456 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-20 16:55:25,465 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.40 KB at sequenceid=21 (bloomFilter=true), to=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/.tmp/cf1/9f0e1e76d73842aaba12686b53b10089 2023-07-20 16:55:25,471 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/.tmp/cf1/9f0e1e76d73842aaba12686b53b10089 as hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1/9f0e1e76d73842aaba12686b53b10089 2023-07-20 16:55:25,477 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/cf1/9f0e1e76d73842aaba12686b53b10089, entries=100, sequenceid=21, filesize=8.3 K 2023-07-20 16:55:25,478 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.40 KB/3480, heapSize ~11.17 KB/11440, currentSize=0 B/0 for 058a0bfd9d8bf39d8c91089a151cb277 in 25ms, sequenceid=21, compaction requested=true 2023-07-20 16:55:25,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/data/default/t1_syncup/058a0bfd9d8bf39d8c91089a151cb277/recovered.edits/24.seqid, newMaxSeqId=24, maxSeqId=17 2023-07-20 16:55:25,493 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:55:25,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 058a0bfd9d8bf39d8c91089a151cb277: 2023-07-20 16:55:25,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed t1_syncup,,1689872067977.058a0bfd9d8bf39d8c91089a151cb277. 2023-07-20 16:55:25,519 INFO [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,44889,1689872108583; all regions closed. 2023-07-20 16:55:25,520 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:25,529 DEBUG [RS:0;jenkins-hbase4:44889] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs 2023-07-20 16:55:25,529 INFO [RS:0;jenkins-hbase4:44889] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C44889%2C1689872108583.meta:.meta(num 1689872109085) 2023-07-20 16:55:25,529 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:25,537 DEBUG [RS:0;jenkins-hbase4:44889] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs 2023-07-20 16:55:25,537 INFO [RS:0;jenkins-hbase4:44889] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C44889%2C1689872108583.jenkins-hbase4.apache.org%2C44889%2C1689872108583.regiongroup-1:(num 1689872113637) 2023-07-20 16:55:25,538 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/WALs/jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:25,550 DEBUG [RS:0;jenkins-hbase4:44889] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/oldWALs 2023-07-20 16:55:25,550 INFO [RS:0;jenkins-hbase4:44889] wal.AbstractFSWAL(1031): Closed WAL: FSHLog jenkins-hbase4.apache.org%2C44889%2C1689872108583.jenkins-hbase4.apache.org%2C44889%2C1689872108583.regiongroup-0:(num 1689872108971) 2023-07-20 16:55:25,550 DEBUG [RS:0;jenkins-hbase4:44889] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:25,550 INFO [RS:0;jenkins-hbase4:44889] regionserver.LeaseManager(133): Closed leases 2023-07-20 16:55:25,550 INFO [RS:0;jenkins-hbase4:44889] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-20 16:55:25,550 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 16:55:25,550 DEBUG [RS:0;jenkins-hbase4:44889] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4acd0e1c to 127.0.0.1:56032 2023-07-20 16:55:25,550 DEBUG [RS:0;jenkins-hbase4:44889] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:25,551 INFO [RS:0;jenkins-hbase4:44889] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44889 2023-07-20 16:55:25,556 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c131003d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:55:25,556 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310027, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:55:25,556 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:44889-0x10183a7c1310034, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/rs/jenkins-hbase4.apache.org,44889,1689872108583 2023-07-20 16:55:25,556 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 16:55:25,556 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:44889-0x10183a7c1310034, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:55:25,556 INFO [zk-event-processor-pool-0] replication.HBaseReplicationEndpoint$PeerRegionServerListener(224): Detected change to peer region servers, fetching updated list 2023-07-20 16:55:25,556 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2/rs 2023-07-20 16:55:25,558 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,44889,1689872108583] 2023-07-20 16:55:25,558 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,44889,1689872108583; numProcessing=1 2023-07-20 16:55:25,560 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /2/draining/jenkins-hbase4.apache.org,44889,1689872108583 already deleted, retry=false 2023-07-20 16:55:25,560 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,44889,1689872108583 expired; onlineServers=0 2023-07-20 16:55:25,560 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39055,1689872108560' ***** 2023-07-20 16:55:25,560 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-20 16:55:25,560 DEBUG [M:0;jenkins-hbase4:39055] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d773c4a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-20 16:55:25,560 INFO [M:0;jenkins-hbase4:39055] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39055,1689872108560 2023-07-20 16:55:25,560 INFO [M:0;jenkins-hbase4:39055] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39055,1689872108560; all regions closed. 2023-07-20 16:55:25,561 DEBUG [M:0;jenkins-hbase4:39055] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-20 16:55:25,561 DEBUG [M:0;jenkins-hbase4:39055] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-20 16:55:25,561 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-20 16:55:25,561 DEBUG [M:0;jenkins-hbase4:39055] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-20 16:55:25,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872108757] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689872108757,5,FailOnTimeoutGroup] 2023-07-20 16:55:25,562 INFO [M:0;jenkins-hbase4:39055] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-20 16:55:25,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872108757] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689872108757,5,FailOnTimeoutGroup] 2023-07-20 16:55:25,562 INFO [M:0;jenkins-hbase4:39055] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-20 16:55:25,563 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/2/master 2023-07-20 16:55:25,563 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/2 2023-07-20 16:55:25,563 INFO [M:0;jenkins-hbase4:39055] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-20 16:55:25,563 DEBUG [M:0;jenkins-hbase4:39055] master.HMaster(1512): Stopping service threads 2023-07-20 16:55:25,563 INFO [M:0;jenkins-hbase4:39055] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-20 16:55:25,564 ERROR [M:0;jenkins-hbase4:39055] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-20 16:55:25,564 INFO [M:0;jenkins-hbase4:39055] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-20 16:55:25,564 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Set watcher on znode that does not yet exist, /2/master 2023-07-20 16:55:25,564 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-20 16:55:25,564 DEBUG [M:0;jenkins-hbase4:39055] zookeeper.ZKUtil(398): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Unable to get data of znode /2/master because node does not exist (not an error) 2023-07-20 16:55:25,564 WARN [M:0;jenkins-hbase4:39055] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-20 16:55:25,564 INFO [M:0;jenkins-hbase4:39055] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-20 16:55:25,565 INFO [M:0;jenkins-hbase4:39055] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-20 16:55:25,565 DEBUG [M:0;jenkins-hbase4:39055] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-20 16:55:25,565 INFO [M:0;jenkins-hbase4:39055] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:55:25,565 DEBUG [M:0;jenkins-hbase4:39055] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:55:25,565 DEBUG [M:0;jenkins-hbase4:39055] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-20 16:55:25,565 DEBUG [M:0;jenkins-hbase4:39055] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:55:25,565 INFO [M:0;jenkins-hbase4:39055] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=26.42 KB heapSize=32.85 KB 2023-07-20 16:55:25,586 INFO [M:0;jenkins-hbase4:39055] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=26.42 KB at sequenceid=310 (bloomFilter=true), to=hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/90bbab9f93754e5888e77a9eab4a8216 2023-07-20 16:55:25,592 DEBUG [M:0;jenkins-hbase4:39055] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/90bbab9f93754e5888e77a9eab4a8216 as hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/90bbab9f93754e5888e77a9eab4a8216 2023-07-20 16:55:25,598 INFO [M:0;jenkins-hbase4:39055] regionserver.HStore(1080): Added hdfs://localhost:33767/user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/90bbab9f93754e5888e77a9eab4a8216, entries=9, sequenceid=310, filesize=5.5 K 2023-07-20 16:55:25,599 INFO [M:0;jenkins-hbase4:39055] regionserver.HRegion(2948): Finished flush of dataSize ~26.42 KB/27051, heapSize ~32.84 KB/33624, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 34ms, sequenceid=310, compaction requested=true 2023-07-20 16:55:25,599 INFO [M:0;jenkins-hbase4:39055] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-20 16:55:25,600 DEBUG [M:0;jenkins-hbase4:39055] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-20 16:55:25,600 DEBUG [WAL-Shutdown-0] wal.FSHLog(489): Closing WAL writer in /user/jenkins/test-data/1ea53b53-dcab-5a2e-ca83-e19cbc60479a/MasterData/WALs/jenkins-hbase4.apache.org,39055,1689872108560 2023-07-20 16:55:25,613 INFO [M:0;jenkins-hbase4:39055] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-20 16:55:25,613 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-20 16:55:25,614 INFO [M:0;jenkins-hbase4:39055] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39055 2023-07-20 16:55:25,617 DEBUG [M:0;jenkins-hbase4:39055] zookeeper.RecoverableZooKeeper(172): Node /2/rs/jenkins-hbase4.apache.org,39055,1689872108560 already deleted, retry=false 2023-07-20 16:55:25,705 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:44889-0x10183a7c1310034, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:25,705 INFO [RS:0;jenkins-hbase4:44889] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,44889,1689872108583; zookeeper connection closed. 2023-07-20 16:55:25,705 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): regionserver:44889-0x10183a7c1310034, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:25,705 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6b8edcf5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6b8edcf5 2023-07-20 16:55:25,705 INFO [Listener at localhost/33489] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-20 16:55:25,905 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:25,905 INFO [M:0;jenkins-hbase4:39055] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39055,1689872108560; zookeeper connection closed. 2023-07-20 16:55:25,905 DEBUG [Listener at localhost/33489-EventThread] zookeeper.ZKWatcher(600): master:39055-0x10183a7c1310033, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-20 16:55:25,907 WARN [Listener at localhost/33489] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-20 16:55:25,912 INFO [Listener at localhost/33489] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 16:55:26,019 WARN [BP-635976616-172.31.14.131-1689872064059 heartbeating to localhost/127.0.0.1:33767] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-20 16:55:26,019 WARN [BP-635976616-172.31.14.131-1689872064059 heartbeating to localhost/127.0.0.1:33767] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-635976616-172.31.14.131-1689872064059 (Datanode Uuid 1b5916d6-dbef-485a-8fd4-49d0508d7da0) service to localhost/127.0.0.1:33767 2023-07-20 16:55:26,021 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/cluster_b91dc339-012d-6b0e-9c91-88d86ff52b70/dfs/data/data7/current/BP-635976616-172.31.14.131-1689872064059] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 16:55:26,022 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/cluster_b91dc339-012d-6b0e-9c91-88d86ff52b70/dfs/data/data8/current/BP-635976616-172.31.14.131-1689872064059] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 16:55:26,029 WARN [Listener at localhost/33489] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-20 16:55:26,033 INFO [Listener at localhost/33489] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 16:55:26,140 WARN [BP-635976616-172.31.14.131-1689872064059 heartbeating to localhost/127.0.0.1:33767] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-20 16:55:26,140 WARN [BP-635976616-172.31.14.131-1689872064059 heartbeating to localhost/127.0.0.1:33767] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-635976616-172.31.14.131-1689872064059 (Datanode Uuid 255edda2-8292-4de8-8e52-18ab644e2dec) service to localhost/127.0.0.1:33767 2023-07-20 16:55:26,141 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/cluster_b91dc339-012d-6b0e-9c91-88d86ff52b70/dfs/data/data5/current/BP-635976616-172.31.14.131-1689872064059] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 16:55:26,141 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/cluster_b91dc339-012d-6b0e-9c91-88d86ff52b70/dfs/data/data6/current/BP-635976616-172.31.14.131-1689872064059] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 16:55:26,143 WARN [Listener at localhost/33489] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-20 16:55:26,150 INFO [Listener at localhost/33489] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 16:55:26,254 WARN [BP-635976616-172.31.14.131-1689872064059 heartbeating to localhost/127.0.0.1:33767] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-20 16:55:26,254 WARN [BP-635976616-172.31.14.131-1689872064059 heartbeating to localhost/127.0.0.1:33767] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-635976616-172.31.14.131-1689872064059 (Datanode Uuid 03416ff6-ea90-4b13-9f8c-c5bf8a981c49) service to localhost/127.0.0.1:33767 2023-07-20 16:55:26,255 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/cluster_b91dc339-012d-6b0e-9c91-88d86ff52b70/dfs/data/data3/current/BP-635976616-172.31.14.131-1689872064059] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 16:55:26,255 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/cluster_b91dc339-012d-6b0e-9c91-88d86ff52b70/dfs/data/data4/current/BP-635976616-172.31.14.131-1689872064059] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 16:55:26,257 WARN [Listener at localhost/33489] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-20 16:55:26,264 INFO [Listener at localhost/33489] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 16:55:26,369 WARN [BP-635976616-172.31.14.131-1689872064059 heartbeating to localhost/127.0.0.1:33767] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-20 16:55:26,369 WARN [BP-635976616-172.31.14.131-1689872064059 heartbeating to localhost/127.0.0.1:33767] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-635976616-172.31.14.131-1689872064059 (Datanode Uuid 5c54da54-6aae-49ea-ad40-5417a6f96449) service to localhost/127.0.0.1:33767 2023-07-20 16:55:26,370 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/cluster_b91dc339-012d-6b0e-9c91-88d86ff52b70/dfs/data/data1/current/BP-635976616-172.31.14.131-1689872064059] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 16:55:26,370 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f5ac3cb-4d34-414a-08e4-b411e503d503/cluster_b91dc339-012d-6b0e-9c91-88d86ff52b70/dfs/data/data2/current/BP-635976616-172.31.14.131-1689872064059] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 16:55:26,402 INFO [Listener at localhost/33489] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 16:55:26,585 INFO [Listener at localhost/33489] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-20 16:55:26,586 INFO [Listener at localhost/33489] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-20 16:55:26,586 WARN [Listener at localhost/33489] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-20 16:55:26,591 INFO [Listener at localhost/33489] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 16:55:26,692 WARN [BP-1126930497-172.31.14.131-1689872057011 heartbeating to localhost/127.0.0.1:38055] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-20 16:55:26,692 WARN [BP-1126930497-172.31.14.131-1689872057011 heartbeating to localhost/127.0.0.1:38055] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1126930497-172.31.14.131-1689872057011 (Datanode Uuid 616ed7c7-88a2-4fe7-8fad-43f3c94c5211) service to localhost/127.0.0.1:38055 2023-07-20 16:55:26,693 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/cluster_1bbf2ea4-f49d-7f4b-90b2-cfdea3662c1b/dfs/data/data3/current/BP-1126930497-172.31.14.131-1689872057011] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 16:55:26,693 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/cluster_1bbf2ea4-f49d-7f4b-90b2-cfdea3662c1b/dfs/data/data4/current/BP-1126930497-172.31.14.131-1689872057011] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 16:55:26,694 WARN [Listener at localhost/33489] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-20 16:55:26,698 INFO [Listener at localhost/33489] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 16:55:26,699 WARN [BP-1126930497-172.31.14.131-1689872057011 heartbeating to localhost/127.0.0.1:38055] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-20 16:55:26,699 WARN [BP-1126930497-172.31.14.131-1689872057011 heartbeating to localhost/127.0.0.1:38055] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1126930497-172.31.14.131-1689872057011 (Datanode Uuid b50e5c74-ff31-44ab-9bcf-8c87552d91c8) service to localhost/127.0.0.1:38055 2023-07-20 16:55:26,700 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/cluster_1bbf2ea4-f49d-7f4b-90b2-cfdea3662c1b/dfs/data/data1/current/BP-1126930497-172.31.14.131-1689872057011] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 16:55:26,700 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c49b3fa0-0892-6604-17dc-52a1b473b4e8/cluster_1bbf2ea4-f49d-7f4b-90b2-cfdea3662c1b/dfs/data/data2/current/BP-1126930497-172.31.14.131-1689872057011] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-20 16:55:26,708 INFO [Listener at localhost/33489] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-20 16:55:26,925 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310027, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Disconnected, path=null 2023-07-20 16:55:26,925 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c1310027, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Disconnected, path=null 2023-07-20 16:55:26,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(630): connection to cluster: 1-0x10183a7c1310027, quorum=127.0.0.1:56032, baseZNode=/2 Received Disconnected from ZooKeeper, ignoring 2023-07-20 16:55:26,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(630): connection to cluster: 1-0x10183a7c1310027, quorum=127.0.0.1:56032, baseZNode=/2 Received Disconnected from ZooKeeper, ignoring 2023-07-20 16:55:26,925 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c131003d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Disconnected, path=null 2023-07-20 16:55:26,925 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c131003d, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Disconnected, path=null 2023-07-20 16:55:26,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(630): connection to cluster: 1-0x10183a7c131003d, quorum=127.0.0.1:56032, baseZNode=/2 Received Disconnected from ZooKeeper, ignoring 2023-07-20 16:55:26,925 DEBUG [ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357-EventThread] zookeeper.ZKWatcher(600): connection to cluster: 1-0x10183a7c131002a, quorum=127.0.0.1:56032, baseZNode=/2 Received ZooKeeper Event, type=None, state=Disconnected, path=null 2023-07-20 16:55:26,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(630): connection to cluster: 1-0x10183a7c131003d, quorum=127.0.0.1:56032, baseZNode=/2 Received Disconnected from ZooKeeper, ignoring 2023-07-20 16:55:26,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(630): connection to cluster: 1-0x10183a7c131002a, quorum=127.0.0.1:56032, baseZNode=/2 Received Disconnected from ZooKeeper, ignoring 2023-07-20 16:55:27,851 INFO [Listener at localhost/33489] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-20 16:55:27,884 INFO [Listener at localhost/33489] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-07-20 16:55:27,909 INFO [Listener at localhost/33489] hbase.ResourceChecker(175): after: replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL#testSyncUpTool Thread=186 (was 10) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:38055 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x2a9add19-SendThread(127.0.0.1:56032) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:369) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1137) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-13-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-16 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x4452d20a sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1298708629.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:33767 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Client (1086431432) connection to localhost/127.0.0.1:33767 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost:33767 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:33767 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-14 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1086431432) connection to localhost/127.0.0.1:38055 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357-SendThread(127.0.0.1:56032) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:369) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1137) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/33489 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost:33767 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: zk-event-processor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@2c3bc8ed java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x4a3c513a sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1298708629.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:38055 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x21b950d4-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: IPC Client (1086431432) connection to localhost/127.0.0.1:33767 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x21b950d4-SendThread(127.0.0.1:56032) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:369) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1137) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-12 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-10 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-11 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-15 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x4a3c513a-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:38055 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x23b44123 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1298708629.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:33767 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x23b44123-SendThread(127.0.0.1:56032) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:369) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1137) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x23b44123-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-9 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x2a9add19-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x4a3c513a-SendThread(127.0.0.1:56032) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:369) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1137) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1086431432) connection to localhost/127.0.0.1:33767 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x4452d20a-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:33767 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x2a9add19 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1298708629.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:33767 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,34183,1689872061357-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305-SendThread(127.0.0.1:56032) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:369) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1137) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost:38055 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-13 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-6-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SinkThread-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x21b950d4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$48/1298708629.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1086431432) connection to localhost/127.0.0.1:38055 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56032@0x4452d20a-SendThread(127.0.0.1:56032) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:369) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1137) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,41127,1689872100966-SendThread(127.0.0.1:56032) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:369) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1137) Potentially hanging thread: RS-EventLoopGroup-6-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-5-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0.replicationSource,1-jenkins-hbase4.apache.org,33103,1689872061305-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) - Thread LEAK? -, OpenFileDescriptor=701 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=172 (was 268), ProcessCount=173 (was 173), AvailableMemoryMB=5447 (was 6717)